Method, device and medium for monitoring safety of unmanned cabin

Abstract

The application discloses a safety monitoring method, device and equipment of an unmanned cabin and a medium, and relates to the technical field of monitoring. The method comprises the following steps: acquiring real-time image data, real-time audio data, real-time environment sensing data and real-time access control data of a target cabin; performing image enhancement processing on the real-time image data to obtain target image data, and performing noise reduction processing on the real-time audio data to obtain target audio data; performing target detection processing on the target image data to obtain cabin visual state information; performing acoustic event recognition processing on the target audio data to obtain cabin audio event information; calculating an environment change rate according to the real-time environment sensing data and historical environment sensing data; and determining a safety monitoring result of the target cabin based on the cabin visual state information, the cabin audio event information, the environment change rate and the real-time access control data. The technical scheme of the embodiment of the application effectively improves the accuracy of the safety monitoring result of the unmanned cabin.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of monitoring technology, and in particular to a method, device, equipment and medium for safety monitoring of unmanned ship cabins. Background Technology

[0002] With the rapid development of unmanned ship technology, the level of automation and intelligence of ships is constantly improving. Safety monitoring of the cabin environment has become a key link in ensuring the normal operation of unmanned ships and the safety of onboard equipment and property.

[0003] Currently, most safety monitoring methods for unmanned vessel cabins rely on single-dimensional information such as video surveillance. These methods are insufficient to comprehensively and accurately reflect the overall safety status of the cabin, which can easily lead to inaccurate monitoring results.

[0004] Therefore, there is an urgent need to propose a new method to solve the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, equipment, and medium for safety monitoring of unmanned vessel cabins, which can effectively improve the accuracy of safety monitoring results for unmanned vessel cabins.

[0006] In a first aspect, embodiments of the present invention provide a safety monitoring method for an unmanned vessel cabin, the method comprising: Acquire real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target compartment; The real-time image data is subjected to image enhancement processing to obtain target image data, and the real-time audio data is subjected to noise reduction processing to obtain target audio data; The target image data is subjected to target detection processing to obtain cabin visual state information; the target audio data is subjected to acoustic event recognition processing to obtain cabin audio event information. The rate of environmental change is calculated based on the real-time environmental perception data and the historical environmental perception data. The safety monitoring results of the target cabin are determined based on the cabin visual status information, cabin audio event information, environmental change rate, and real-time access control data.

[0007] The technical solution of this invention first acquires real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target cabin, realizing full-dimensional data collection for the safety monitoring of the target cabin. This avoids the one-sidedness of monitoring from a single data dimension and lays a complete and comprehensive raw data foundation for subsequent judgment of safety monitoring results, thereby improving the scientificity and accuracy of the safety monitoring results. Next, image enhancement processing is performed on the real-time image data to obtain target image data, and noise reduction processing is performed on the real-time audio data to obtain target audio data. This effectively removes invalid interference information and enhances effective feature information, enabling the obtained target image data to more accurately reflect the real visual scene of the cabin. This provides high-quality, high-recognition visual foundation data for subsequent target detection, significantly reducing the probability of missed and false judgments in the subsequent recognition process. Simultaneously, the obtained target audio data can clearly restore the real sound environment of the cabin, providing high signal-to-noise ratio and high-definition acoustic foundation data for acoustic event recognition, effectively improving the accuracy and efficiency of recognition, and laying a solid data quality foundation for the scientific judgment of the final safety monitoring results. Subsequently, target image data is processed for target detection to obtain cabin visual status information; target audio data is processed for acoustic event recognition to obtain cabin audio event information. This transforms visual image data into effective information that can be directly used for safety assessment, avoiding the subjectivity and lag of manual identification. Simultaneously, intelligent analysis of the cabin's acoustic environment is achieved, accurately capturing acoustic anomalies that are difficult to cover by visual monitoring, compensating for information blind spots in single-dimensional monitoring, and making the subsequent depiction of cabin safety status more comprehensive and three-dimensional, further improving the scientific rigor and accuracy of the overall safety monitoring results. Then, the environmental change rate is calculated based on real-time and historical environmental perception data, compensating for information blind spots in single-dimensional monitoring. This not only provides a data foundation for determining the safety monitoring results of the target cabin but also improves the scientific rigor and accuracy of the overall safety monitoring results. Finally, the safety monitoring results of the target cabin are determined based on cabin visual status information, cabin audio event information, environmental change rate, and real-time access control data. This avoids safety monitoring deviations caused by single data anomalies or misjudgments, improving the accuracy of safety monitoring results and effectively enhancing the intelligence and efficiency of cabin safety monitoring. Therefore, the technical solution of the present invention solves the problem that the prior art is unable to comprehensively and accurately reflect the overall safety status of the cabin, thus leading to inaccurate monitoring results.

[0008] Secondly, embodiments of the present invention also provide a safety monitoring device for unmanned ship cabins, the device comprising: The acquisition module is used to acquire real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target compartment. The processing module is used to perform image enhancement processing on the real-time image data to obtain target image data, and to perform noise reduction processing on the real-time audio data to obtain target audio data; The recognition module is used to perform target detection processing on the target image data to obtain cabin visual state information; and to perform acoustic event recognition processing on the target audio data to obtain cabin audio event information. The calculation module is used to calculate the rate of environmental change based on the real-time environmental perception data and the historical environmental perception data; The monitoring module is used to determine the safety monitoring results of the target cabin based on the cabin visual status information, the cabin audio event information, the environmental change rate, and the real-time access control data.

[0009] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the safety monitoring method for unmanned ship cabins according to any embodiment of the present invention.

[0010] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, implement the safety monitoring method for unmanned ship cabins described in any embodiment of the present invention.

[0011] Fifthly, this application provides a computer program product including computer instructions that, when executed on a computer, cause the computer to perform the safety monitoring method for unmanned ship cabins as provided in the first aspect.

[0012] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the safety monitoring device for the unmanned vessel's cabin, or it may be packaged separately from the processor of the safety monitoring device for the unmanned vessel's cabin; this application does not impose any limitations on this.

[0013] The descriptions of the second, third, fourth, and fifth aspects in this application can be referred to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0014] In this application, the name of the aforementioned safety monitoring device for the unmanned vessel cabin does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0015] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a safety monitoring method for an unmanned ship cabin provided in an embodiment of the present invention; Figure 2 A flowchart illustrating another method for safety monitoring of unmanned ship cabins provided in an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a safety monitoring device for an unmanned ship cabin provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0019] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0021] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0022] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, without conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0023] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0024] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0025] Figure 1 This is a flowchart illustrating a safety monitoring method for unmanned vessel compartments according to an embodiment of the present invention. This embodiment is applicable to situations requiring safety monitoring of various compartments of an unmanned vessel. The method can be executed by a safety monitoring device for the unmanned vessel compartments, which can be implemented using software and / or hardware. For example, the device can be an electronic device. (Reference) Figure 1 The safety monitoring method for unmanned ship cabins in this embodiment specifically includes the following steps: Step 110: Acquire real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target compartment.

[0026] Specifically, the target compartment is the specific compartment within the unmanned surface vessel (USV) selected as the monitoring object, and it serves as the corresponding entity for the collection, analysis, and result determination of all monitoring data. Real-time image data consists of raw images or video streams acquired in real-time, reflecting the current visual scene of the target compartment. Real-time audio data consists of raw audio sequences acquired in real-time, reflecting the current sound environment of the target compartment, including unprocessed sound information such as environmental noise and various sound sources. Real-time environmental perception data consists of current environmental physical quantity data for the target compartment, such as parameters related to the compartment environment, including temperature, humidity, smoke concentration, harmful gas content, and illuminance. Real-time access control data consists of dynamic data collected in real-time related to the control of compartment entrances and exits, such as access control status, access control anomaly alarms, and authorized personnel entry and exit records.

[0027] In practice, real-time image data of the target cabin can be obtained through image acquisition devices (such as cameras) installed in the target cabin, real-time audio data of the target cabin can be obtained through audio acquisition devices (such as microphones) installed in the target cabin, real-time environmental perception data of the target cabin can be obtained through environmental sensors (such as temperature sensors, smoke sensors, etc.) installed in the target cabin, and real-time access control data of the cabin can be obtained by the cabin access control system.

[0028] In this embodiment, the above steps enable the collection of comprehensive data for the safety monitoring of the target compartment, avoiding the one-sidedness of monitoring from a single data dimension. This lays a complete and comprehensive foundation of raw data for the subsequent determination of the safety monitoring results, thereby improving the scientific nature and accuracy of the safety monitoring results.

[0029] Step 120: Perform image enhancement processing on the real-time image data to obtain the target image data, and perform noise reduction processing on the real-time audio data to obtain the target audio data.

[0030] Specifically, the target image data is the image data obtained after performing image enhancement processing on real-time image data. The target audio data is the audio data obtained after performing noise reduction processing on real-time audio data.

[0031] In practice, real-time image data can first undergo preprocessing operations such as format normalization and resolution unification to ensure its basic usability. Then, image enhancement algorithms (such as multi-scale retinal enhancement, adaptive histogram equalization, gamma correction, and Laplacian sharpening) are used to enhance the preprocessed real-time image data to obtain the target image data. Simultaneously, real-time audio data undergoes preprocessing such as sampling rate normalization, single / multi-channel unification, and DC component removal to calibrate the basic parameters of the audio data. Next, unsupervised noise estimation algorithms (such as minimum value statistics) are used to extract background noise feature parameters (such as noise spectrum and energy distribution) from silent or low signal-to-noise ratio segments of the audio data to obtain the corresponding noise features. Subsequently, based on the extracted noise features, denoising algorithms (such as a combination of bandpass filtering and spectral subtraction, Wiener filtering, and wavelet thresholding) are used to denoise the audio data to eliminate low-frequency engine noise and ocean wave background noise, thereby obtaining clear target audio data.

[0032] In this embodiment, the above steps effectively eliminate invalid interference information and enhance effective feature information, enabling the obtained target image data to more accurately reflect the real visual scene of the cabin. This provides high-quality, high-recognition visual foundation data for subsequent target detection, significantly reducing the probability of missed and false positives in the subsequent recognition process. Simultaneously, the obtained target audio data clearly reproduces the real sound environment of the cabin, providing high signal-to-noise ratio, high-definition acoustic foundation data for acoustic event recognition, effectively improving the accuracy and efficiency of recognition, and laying a solid data quality foundation for the scientific determination of the final safety monitoring results. Furthermore, the above steps are adaptable to various environmental interference scenarios within the cabin, flexibly responding to sudden changes in light and noise types, effectively improving the scenario adaptability and robustness of the data processing stage.

[0033] Step 130: Perform target detection processing on the target image data to obtain cabin visual state information; perform acoustic event recognition processing on the target audio data to obtain cabin audio event information.

[0034] Specifically, the cabin visual state information is a set of information generated after target detection processing of the target image data. It comprehensively reflects the real-time visual state of the target cabin. For example, the cabin visual state information may include the detected target category, the confidence score of the corresponding target, the target location bounding box coordinates, and the target quantity statistics. The cabin audio event information is a set of information generated after acoustic event recognition processing of the target audio data. It reflects the current sound-level event situation in the target cabin. For example, the cabin audio event information may include the sound event category, the probability of the event occurring, the duration of the event, and the time of the event.

[0035] In the specific implementation, the target image data is input into a pre-trained target detection model to obtain the visual state information of the cabin. Simultaneously, Mel-frequency cepstral analysis is used to convert the target audio data into a Mel spectrogram, which is then input into a pre-trained acoustic event recognition model to obtain cabin audio event information. The pre-trained target detection model is obtained by supervising the training of deep learning models (such as YOLOv5s, SSD-MobileNet, Faster R-CNN, etc.) with a large amount of historical cabin scene image data and their corresponding target categories and bounding box location annotations. The pre-trained acoustic event recognition model is obtained by training a deep learning model (such as CNN+LSTM, Transformer, CRNN, etc.) with a large amount of historical cabin scene audio data, extracting its Mel spectrogram as features, and using the corresponding sound event category labels (such as "personnel conversation," "equipment alarm," etc.) as supervision signals.

[0036] In this embodiment, through the above steps, visual image data can be transformed into effective information that can be directly used for safety assessment, avoiding the subjectivity and lag of manual identification; at the same time, it enables intelligent analysis of the cabin's acoustic environment, accurately captures acoustic anomaly signals that are difficult to cover by visual monitoring, makes up for the information blind spots of single-dimensional monitoring, and makes the subsequent depiction of the cabin's safety status more comprehensive and three-dimensional, further improving the scientificity and accuracy of the overall safety monitoring results.

[0037] Step 140: Calculate the environmental change rate based on real-time environmental perception data and historical environmental perception data.

[0038] Specifically, historical environmental perception data refers to environmental perception data stored over a past period (such as minutes, hours, or days), used to compare with current data to determine trends and anomalies. The rate of environmental change is a quantitative indicator used to describe the drastic change of a specific environmental parameter or set of environmental parameters (such as temperature or smoke concentration) per unit of time.

[0039] In the specific implementation, firstly, historical data corresponding to the real-time environmental sensing data is selected to ensure that the parameter types are consistent (e.g., both are temperature, smoke concentration, harmful gas content, etc.) and the collection dimensions are unified (same sensor, same monitoring point). Next, missing or abnormal historical data is supplemented using methods such as linear interpolation, or invalid data is directly removed. Then, historical data of a preset time period (e.g., data from the 5 seconds prior to the current moment) is selected, and the average value of the historical data within this period is calculated as the historical time period baseline. Based on this, the environmental change rate is calculated using the formula: Environmental Change Rate = (Real-time Environmental Sensing Data - Historical Time Period Baseline Value) / Preset Time Period. Historical data must be collected at equal time intervals (e.g., once per second) to ensure the objectivity of the average value calculation and avoid baseline value deviation caused by uneven data collection density.

[0040] Optionally, to reduce implementation complexity, the formula for calculating the rate of environmental change can also be: Rate of environmental change = |Real-time environmental sensing data - Historical environmental sensing data (i.e., environmental sensing data at the preset historical sampling time)| / (Current time - Preset historical sampling time).

[0041] In this embodiment, the above steps make up for the information blind spots of single-dimensional monitoring, which not only provides a data foundation for the subsequent determination of the safety monitoring results of the target compartment, but also improves the scientificity and accuracy of the overall safety monitoring results.

[0042] Step 150: Determine the safety monitoring results of the target cabin based on cabin visual status information, cabin audio event information, environmental change rate, and real-time access control data.

[0043] Specifically, the safety monitoring results are a comprehensive conclusion on the current safety status of the target cabin based on cabin visual status information, cabin audio event information, environmental change rate and real-time access control data. For example, the safety monitoring results may include the safety monitoring risk level (such as safe, level 1 warning, level 2 warning, danger, etc.) and the corresponding targeted handling methods (such as alarm, marking, power outage, etc.) for each risk level.

[0044] In practice, cabin visual status information, cabin audio event information, environmental change rate, and real-time access control data can be input into a pre-trained safety monitoring model to obtain the safety monitoring results for the target cabin. The safety monitoring model is a dedicated multi-feature fusion judgment model adapted for cabin safety monitoring. This model is based on historical cabin visual status information, historical audio event information, historical environmental change rate, historical access control data, and corresponding historical cabin safety status conclusions (such as risk level and handling methods). It is obtained by scenario-based training, optimization, and validation of deep learning fusion models (such as CNN-LSTM fusion models, Transformer multi-feature fusion models, and lightweight multi-dimensional decision models). After the safety monitoring results for the target cabin are obtained, the results can be pushed to the staff's terminal devices in real time to remind them to take appropriate action.

[0045] It should be noted that the method provided in this embodiment of the invention can monitor one or more target compartments, and can simultaneously carry out safety monitoring of multiple compartments and obtain the corresponding safety monitoring results of each compartment. Furthermore, it can push all monitoring results to the staff's terminal devices in real time, so as to remind the staff to take corresponding measures in a timely manner for the safety status of different compartments.

[0046] In this embodiment, the above steps can avoid safety monitoring deviations caused by single data anomalies or misjudgments, improve the accuracy of safety monitoring results, and effectively enhance the intelligence and efficiency of cabin safety monitoring.

[0047] The safety monitoring method for unmanned vessel cabins provided in this invention first acquires real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target cabin. This achieves multi-dimensional data collection for safety monitoring of the target cabin, avoiding the one-sidedness of monitoring from a single data dimension. It lays a complete and comprehensive foundation of raw data for subsequent judgment of safety monitoring results, thereby improving the scientific rigor and accuracy of the safety monitoring results. Next, image enhancement processing is performed on the real-time image data to obtain target image data, and noise reduction processing is performed on the real-time audio data to obtain target audio data. This effectively removes invalid interference information and enhances effective feature information, enabling the obtained target image data to more accurately reflect the real visual scene of the cabin. This provides high-quality, high-recognition visual foundation data for subsequent target detection, significantly reducing the probability of missed and false judgments in the subsequent identification process. Simultaneously, the obtained target audio data clearly restores the real sound environment of the cabin, providing high signal-to-noise ratio and high-definition acoustic foundation data for acoustic event recognition, effectively improving the accuracy and efficiency of recognition, and laying a solid data quality foundation for the scientific judgment of the final safety monitoring results. Subsequently, target image data is processed for target detection to obtain cabin visual status information; target audio data is processed for acoustic event recognition to obtain cabin audio event information. This transforms visual image data into effective information that can be directly used for safety assessment, avoiding the subjectivity and lag of manual identification. Simultaneously, intelligent analysis of the cabin's acoustic environment is achieved, accurately capturing acoustic anomalies that are difficult to cover by visual monitoring, compensating for information blind spots in single-dimensional monitoring, and making the subsequent depiction of cabin safety status more comprehensive and three-dimensional, further improving the scientific rigor and accuracy of the overall safety monitoring results. Then, the environmental change rate is calculated based on real-time and historical environmental perception data, compensating for information blind spots in single-dimensional monitoring. This not only provides a data foundation for determining the safety monitoring results of the target cabin but also improves the scientific rigor and accuracy of the overall safety monitoring results. Finally, the safety monitoring results of the target cabin are determined based on cabin visual status information, cabin audio event information, environmental change rate, and real-time access control data. This avoids safety monitoring deviations caused by single data anomalies or misjudgments, improving the accuracy of safety monitoring results and effectively enhancing the intelligence and efficiency of cabin safety monitoring. Therefore, the technical solution of the present invention solves the problem that the prior art is unable to comprehensively and accurately reflect the overall safety status of the cabin, thus leading to inaccurate monitoring results.

[0048] Figure 2 This is a flowchart illustrating another safety monitoring method for an unmanned vessel cabin provided by an embodiment of the present invention. This embodiment is a specific implementation based on the above embodiment. In this embodiment, the method may further include: Step 210: Acquire real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target compartment.

[0049] Step 211: Perform image enhancement processing on the real-time image data to obtain the target image data, and perform noise reduction processing on the real-time audio data to obtain the target audio data.

[0050] Optionally, real-time environmental perception data includes real-time ambient light intensity.

[0051] Furthermore, before performing image enhancement processing on the real-time image data to obtain the target image data, the method further includes: determining whether the real-time ambient light intensity is less than the preset light intensity; if the real-time ambient light intensity is less than the preset light intensity, then performing image enhancement processing on the real-time image data to obtain the target image data; if the real-time ambient light intensity is not less than the preset light intensity, then determining the real-time image data as the target image data.

[0052] Specifically, real-time ambient light intensity is a physical quantity data reflecting the intensity of ambient light at the current moment, collected in real time by light sensors (such as photodiodes, photoresistors, or lux meters) deployed in the monitoring area of ​​the target cabin. Its commonly used unit of measurement is lux (Lux). Preset light intensity is a light intensity threshold set in advance according to actual conditions or needs, for example: preset light intensity is 30 Lux.

[0053] In practice, before performing image enhancement processing on real-time image data to obtain target image data, it can be determined whether the real-time ambient light intensity is less than the preset light intensity. If the real-time ambient light intensity is less than the preset light intensity, it indicates that the target cabin monitoring area is in a dim or low-light environment. At this time, image enhancement processing can be performed on the real-time image data to obtain target image data. If the real-time ambient light intensity is not less than the preset light intensity, it indicates that the lighting conditions of the target cabin monitoring area meet the requirements for image acquisition and subsequent recognition. At this time, the real-time image data can be directly determined as target image data.

[0054] In this embodiment, the above steps effectively improve the visual clarity and identifiability of low-light images, providing high-quality target image data for the subsequent extraction of visual status information of the cabin. This ensures the accuracy of visual monitoring under different lighting conditions and avoids target omissions and misjudgments due to insufficient light. Simultaneously, images with adequate lighting conditions can directly use the original data without additional enhancement processing. This effectively reduces unnecessary computational power consumption and image processing time, while improving the overall processing efficiency of the visual monitoring workflow, thus meeting the real-time requirements of cabin safety monitoring.

[0055] Optionally, to eliminate hardware dependencies and reduce deployment costs, before performing image enhancement processing on the real-time image data to obtain the target image data, the real-time image data can be directly analyzed using the global average brightness calculation method or the regional brightness ratio calculation method to determine whether the target cabin monitoring area is in a dim or low-light environment. If the determination result is that it is in a dim or low-light environment, image enhancement processing is performed on the real-time image data to obtain the target image data; if the determination result is that it is not in a dim or low-light environment, the real-time image data is directly determined as the target image data.

[0056] Step 212: Perform target detection processing on the target image data to obtain cabin visual state information; perform acoustic event recognition processing on the target audio data to obtain cabin audio event information.

[0057] Step 213: Calculate the environmental change rate based on real-time environmental perception data and historical environmental perception data.

[0058] Step 214: Determine the general risk monitoring results of the target cabin based on the cabin visual status information, cabin audio event information, and environmental change rate.

[0059] Specifically, the general risk monitoring results are safety risk conclusions drawn from the general, non-personnel intrusion physical event safety monitoring of the target cabin based on cabin visual status information, cabin audio event information, and environmental change rate. For example, the general risk monitoring results may include general risk levels (such as no risk, level 1 risk, level 2 risk, etc.) and corresponding risk types (such as fire alarm, equipment abnormality, electrical short circuit, etc.).

[0060] In practice, cabin visual state information, cabin audio event information, and environmental change rate can be input into a pre-trained general risk monitoring model to obtain the general risk monitoring results for the target cabin. The general risk monitoring model is obtained by training a deep learning model based on historical cabin visual state information, historical cabin audio event information, historical environmental change rate, and corresponding historical general risk monitoring results.

[0061] In this embodiment, the above steps effectively improve the comprehensiveness, accuracy, and reliability of general risk monitoring results.

[0062] Optionally, the cabin visual status information includes visual type and recognition confidence; the cabin audio event information includes audio event category and category confidence.

[0063] Furthermore, step 214 may specifically include: determining candidate general risk types based on visual type, audio event category, environmental change rate, and risk feature library; mapping the environmental change rate to obtain environmental change mapping value; calculating the risk reliability of candidate general risk types based on the recognition confidence corresponding to visual type, the category confidence corresponding to audio event category, and the environmental change mapping value; and determining the general risk monitoring results of the target cabin based on the candidate general risk types and their corresponding risk reliability.

[0064] Specifically, visual type refers to the identified target category extracted from the cabin's visual state information, such as "personnel," "smoke," "open flame," "equipment," and "doors and windows (open)." Recognition confidence is the probability quantification value corresponding to the visual type output by the target detection model, used to measure the reliability of the visual recognition. Audio event category refers to the identified sound event type extracted from the cabin's audio event information, such as "personnel conversation," "equipment malfunction alarm," "glass breaking sound," "metal impact sound," and "explosion sound." Category confidence is the probability quantification value corresponding to the audio event category output by the acoustic event recognition model. The risk feature library is a dedicated rule library pre-built according to actual conditions or needs, storing the mapping relationship between various common risk types in the cabin (such as fire alarms, short circuits, etc.) and multi-dimensional judgment features. Its core is defining the feature combination of visual type, audio event category, and environmental change rate threshold associated with each risk type, which is the core basis for matching candidate common risk types. Candidate general risk types are one or more potential general risk types selected by comparing and matching the currently extracted visual types, audio event categories, and environmental change rates with rules in the risk feature library. The environmental change mapping value is a dimensionless quantified value obtained after standardizing and normalizing the environmental change rate. Risk reliability is a comprehensive confidence score calculated for each candidate general risk type by fusing its corresponding identification confidence, category confidence, and environmental change mapping value, and considering the weight of each piece of evidence. This score quantifies the degree of matching between the candidate risk type and the actual safety status of the target cabin.

[0065] In practice, candidate general risk types can be obtained by first matching visual type, audio event category, environmental change rate, and rules in the risk feature library. For example, if the visual type includes "smoke" and the environmental temperature change rate is greater than 5℃ / min, the candidate general risk type can be determined as "fire alarm". If the visual type includes "smoke" or "open flame", the audio event category is "explosion sound" or "crackling sound", and the environmental temperature change rate is greater than 5℃ / min, the candidate general risk type can be determined as "fire alarm". If the visual type includes "liquid stains" or "gas mist", the audio event category is "gas hissing sound" or "liquid dripping sound", and the environmental air pressure change rate is greater than 2kPa / min, the candidate general risk type can be determined as "pipeline leakage". If the visual type includes "water accumulation area" or "water seepage through doors and windows", the audio event category is "water impact sound", and the environmental humidity change rate is greater than 4% / min, the candidate general risk type can be determined as "water ingress into the cabin".

[0066] Then, the environmental change rate is mapped using an environmental change mapping table to obtain the environmental change mapping value. The environmental change mapping table is a pre-defined table that stores the actual values ​​of various environmental change rates (such as temperature, air pressure, and humidity change rates) and their corresponding dimensionless mapping values, based on actual conditions or needs. For example, when the temperature change rate is 3.5℃ / minute, the corresponding environmental change mapping value is 0.8.

[0067] Next, based on the recognition confidence level corresponding to the visual type, the category confidence level corresponding to the audio event category, and the environmental change mapping value, the risk reliability of the candidate general risk type is calculated. The specific calculation formula is: Risk reliability of candidate general risk type = Visual weight × Recognition confidence level + Audio weight × Category confidence level + Environmental weight × Environmental change mapping value. Wherein, visual weight, environmental weight, and audio weight are pre-set weight coefficients based on actual conditions or needs, and the sum of the three values ​​is 1.

[0068] Furthermore, if the matching rules in the risk feature library only associate features with certain dimensions (not visual, audio, or all environmental dimensions), then the risk reliability calculation only applies a weighted average to the dimensions involved in the rule; dimensions not involved are not included in the calculation. For example, if the matching rule is: when the visual type contains "smoke" and the environmental temperature change rate is greater than 5℃ / min, then the candidate general risk type can be determined as "fire alarm." In this case, the risk reliability of the candidate general risk type = visual weight × recognition confidence + environmental weight × environmental change mapping value.

[0069] Finally, based on the candidate general risk types and their corresponding risk reliability, the general risk monitoring results for the target compartment are determined. Specifically, first, the risk reliability is queried in the reliability-hazard level mapping table to obtain the corresponding risk level. Then, the matched hazard levels are associated with the candidate general risk types to obtain a summary result of level and risk type. For example, if the hazard level is Level 1 and the candidate general risk type is fire alarm, the summary result is Level 1 fire alarm. Based on this summary result, the corresponding emergency response method is matched in the preset risk handling rule table. Finally, the summary result and the matched response method are integrated to obtain the general risk monitoring results for the target compartment. The reliability-hazard level mapping table is a pre-set matching table of risk reliability and hazard level according to actual conditions or needs. For example, if the risk reliability is > 0.9, the hazard level is Level 1 (the highest hazard level); if 0.8 < risk reliability ≤ 0.9, the hazard level is Level 2; and if the risk reliability ≤ 0.8, the hazard level is Level 3. The risk handling rules table is a matching table of the summary results of risk levels and types pre-set according to actual situation or needs and the emergency response methods. For example, the response method corresponding to a level one fire alarm is alarm + power outage.

[0070] In this embodiment, the accuracy and reliability of general risk monitoring results are effectively improved through the above steps, including the multi-dimensional data fusion and quantitative analysis.

[0071] Step 215: Determine the intrusion risk monitoring results of the target compartment based on the compartment visual status information, compartment audio event information, and real-time access control data.

[0072] Specifically, the intrusion risk monitoring results are security risk conclusions drawn after conducting security monitoring of personnel intrusion events in the target cabin based on cabin visual status information, cabin audio event information, and real-time access control data. For example, the intrusion risk monitoring results may include intrusion risk level (such as no risk, level 1 risk, level 2 risk, etc.) and intrusion judgment conclusion (such as intrusion or no intrusion).

[0073] In practice, cabin visual status information, cabin audio event information, and real-time access control data can be input into a pre-trained intrusion risk monitoring model to obtain the intrusion risk monitoring results for the target cabin. The intrusion risk monitoring model is a deep learning model trained based on historical cabin visual status information, historical cabin audio event information, historical access control data, and corresponding historical intrusion risk monitoring results.

[0074] In this embodiment, the accuracy of the target compartment intrusion risk monitoring results is effectively improved through the above steps.

[0075] Further, step 215 may specifically include: determining whether the real-time access control data is in an abnormal door opening state; if the real-time access control data is in an abnormal door opening state, then determining the intrusion risk monitoring result based on the cabin visual state information and cabin audio event information; if the real-time access control data is not in an abnormal door opening state, then determining the intrusion risk monitoring result as no intrusion risk.

[0076] Specifically, abnormal door opening status refers to the cabin door opening status represented by real-time access control data that has not gone through a legitimate authorization process, such as opening the door without authorization by swiping a card, opening the door by force, or opening the door after the access control permission has expired. Normal door opening status refers to the cabin door opening status represented by real-time access control data that has been completed through a legitimate authorization process, such as opening the door by authorized personnel swiping a card, or opening the door remotely by a legitimate command from the backend.

[0077] In practice, the collected real-time access control data of the target cabin can first be analyzed to extract core information such as access control opening authorization verification records, operation triggering methods, and status identifiers. This information is then compared with preset normal access control authorization rules to determine whether the real-time access control data indicates an abnormal door opening state. The preset normal access control authorization rules are a pre-defined set of criteria and standards for determining the compliance of cabin access control opening behavior. For example, a preset normal access control authorization rule might state that when the access control is opened, authorized personnel must complete facial recognition / card swipe verification, and the verification result must be successful.

[0078] If the real-time access control data indicates an abnormal door opening, the intrusion risk monitoring result is determined based on the cabin's visual state information and audio event information. Specifically, rule matching can be performed based on the cabin's visual state information, audio event information, and an intrusion risk feature database to determine the intrusion risk monitoring result. An example of a matching rule is: if the visual type contains "human figure," or the audio event category contains "footsteps" or "speaking," then the intrusion risk monitoring result is determined to be "intrusion." The intrusion risk feature database is a dedicated rule library pre-built according to actual conditions or needs, storing rules for determining personnel intrusion types. Its core is defining the mapping and matching relationship between features such as visual type and audio event category and the intrusion risk monitoring result. If the real-time access control data does not indicate an abnormal door opening, the intrusion risk monitoring result is determined to be no intrusion risk.

[0079] In this embodiment, the above steps enable pre-screening of access control status, effectively reducing invalid calculations and improving monitoring efficiency. At the same time, it focuses on high-risk scenarios to conduct precise analysis, improving the accuracy of intrusion risk monitoring results. Moreover, the overall hierarchical analysis logic is simple and efficient, and the system is easy to implement and deploy.

[0080] Optionally, to further improve the accuracy of intrusion risk monitoring results, the same quantitative assessment method used to calculate risk reliability to determine general risk monitoring results can be adopted to determine the intrusion risk monitoring results of the target compartment.

[0081] Furthermore, after step 215, the method further includes: determining whether the intrusion risk monitoring result indicates the existence of an intrusion risk; if the intrusion risk monitoring result indicates the existence of an intrusion risk; performing behavior recognition processing on the target image data to obtain a behavior recognition result; and updating the intrusion risk monitoring result based on the behavior recognition result to obtain an updated intrusion risk monitoring result.

[0082] Specifically, the behavior recognition result is the judgment conclusion output after the behavior recognition processing of the target image data. For example, the behavior recognition result may include standing upright, bending over to pick a lock, falling to the ground and not moving, loitering, accidentally touching the access control, normal operation, etc.

[0083] In practice, after obtaining the intrusion risk monitoring results, it can be first determined whether the intrusion risk monitoring results indicate the presence of an intrusion risk. If there is no intrusion risk, no further action is required, and the original monitoring results remain unchanged. If there is an intrusion risk, a behavior recognition model (such as the YOLOv8-Pose model) is used to perform behavior recognition processing on the target image data to obtain behavior recognition results. The intrusion risk monitoring results are then updated based on the behavior recognition results to obtain the updated intrusion risk monitoring results. Specifically, if the behavior recognition results indicate a pre-set illegal behavior based on actual circumstances or needs (such as lock picking, forced entry, climbing over fences, etc.), the original intrusion risk monitoring results remain unchanged, or the risk level can be increased by one level; if the behavior recognition results do not indicate a pre-set illegal behavior, the original intrusion risk monitoring results are updated to indicate no intrusion risk.

[0084] In this embodiment, the secondary behavior verification is carried out through the above steps, which greatly reduces the probability of false intrusion risk and enhances the reliability of intrusion risk monitoring results.

[0085] Step 216: Integrate the general risk monitoring results and the intrusion risk monitoring results to obtain the security monitoring results of the target compartment.

[0086] In practice, after obtaining the general risk monitoring results and the intrusion risk monitoring results, the two types of monitoring results are merged and integrated to comprehensively determine and generate the overall security monitoring result of the target compartment. For example, if the general risk monitoring result is no risk and the intrusion risk monitoring result is a level one intrusion risk, then the security monitoring result is that the target compartment has a level one intrusion risk, but no general security risk.

[0087] In this embodiment, the above steps effectively improve the accuracy and comprehensiveness of the overall safety monitoring results of the target compartment.

[0088] Step 217: Send the safety monitoring results of the target compartment to the staff's terminal.

[0089] Specifically, the terminals used by staff are electronic devices used by personnel involved in cabin safety management that can receive safety monitoring results, such as computer terminals for management back-end systems, mobile phone / tablet terminals for mobile office work, and monitoring terminals for security duty.

[0090] In practice, after obtaining the safety monitoring results of the target compartment, they can be transmitted to the staff's terminal via satellite or / and wireless network, so that the staff can promptly grasp the safety status of the compartment and take subsequent actions. In addition, if the risk level of the safety monitoring results is a preset high-risk level (such as level one), alarm and visual deterrence measures (such as high-volume alarms, flashing lights, etc.) will be triggered simultaneously when transmitting the results, and the corresponding related monitoring data will be transmitted back at the same time, thereby achieving a ship-shore coordinated safety response effect.

[0091] The safety monitoring method for unmanned vessel cabins provided in this invention first acquires real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target cabin. This achieves multi-dimensional data collection for safety monitoring of the target cabin, avoiding the one-sidedness of monitoring from a single data dimension. It lays a complete and comprehensive foundation of raw data for subsequent judgment of safety monitoring results, thereby improving the scientific rigor and accuracy of the safety monitoring results. Next, image enhancement processing is performed on the real-time image data to obtain target image data, and noise reduction processing is performed on the real-time audio data to obtain target audio data. This effectively removes invalid interference information and enhances effective feature information, enabling the obtained target image data to more accurately reflect the real visual scene of the cabin. This provides high-quality, high-recognition visual foundation data for subsequent target detection, significantly reducing the probability of missed and false judgments in the subsequent identification process. Simultaneously, the obtained target audio data clearly restores the real sound environment of the cabin, providing high signal-to-noise ratio and high-definition acoustic foundation data for acoustic event recognition, effectively improving the accuracy and efficiency of recognition, and laying a solid data quality foundation for the scientific judgment of the final safety monitoring results. Subsequently, target image data is processed for target detection to obtain cabin visual status information; target audio data is processed for acoustic event recognition to obtain cabin audio event information. This transforms visual image data into effective information that can be directly used for safety assessment, avoiding the subjectivity and lag of manual identification. Simultaneously, intelligent analysis of the cabin's acoustic environment is achieved, accurately capturing acoustic anomalies that are difficult to cover by visual monitoring, compensating for information blind spots in single-dimensional monitoring, and making the subsequent depiction of cabin safety status more comprehensive and three-dimensional, further improving the scientific rigor and accuracy of the overall safety monitoring results. Then, based on real-time and historical environmental perception data, the environmental change rate is calculated, compensating for information blind spots in single-dimensional monitoring. This not only provides a data foundation for determining the safety monitoring results of the target cabin but also improves the scientific rigor and accuracy of the overall safety monitoring results. Finally, based on cabin visual status information, cabin audio event information, and environmental change rate, the general risk monitoring results of the target cabin are determined, effectively improving the comprehensiveness, accuracy, and reliability of the general risk monitoring results. Based on visual status information, audio event information, and real-time access control data of the cabin, the intrusion risk monitoring results of the target cabin are determined, effectively improving the accuracy of the intrusion risk monitoring results. By integrating general risk monitoring results and intrusion risk monitoring results, the security monitoring results of the target cabin are obtained, effectively improving the accuracy and comprehensiveness of the overall security monitoring results of the target cabin. The security monitoring results of the target cabin are sent to the staff's terminals so that staff can promptly grasp the cabin's security status and take subsequent actions. Therefore, the technical solution of this invention solves the problem in the prior art that it is difficult to comprehensively and accurately reflect the overall security status of the cabin, thus leading to inaccurate monitoring results.

[0092] Figure 3 This is a schematic diagram of the structure of a safety monitoring device for an unmanned vessel cabin provided in an embodiment of the present invention. This device and the safety monitoring method for unmanned vessel cabins in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the safety monitoring device for unmanned vessel cabins, please refer to the embodiments of the safety monitoring method for unmanned vessel cabins described above.

[0093] like Figure 3 As shown, the device includes: The acquisition module 310 is used to acquire real-time image data, real-time audio data, real-time environmental perception data and real-time access control data of the target compartment; The processing module 320 is used to perform image enhancement processing on the real-time image data to obtain target image data, and to perform noise reduction processing on the real-time audio data to obtain target audio data. The recognition module 330 is used to perform target detection processing on the target image data to obtain cabin visual state information; and to perform acoustic event recognition processing on the target audio data to obtain cabin audio event information. Calculation module 340 is used to calculate the rate of environmental change based on the real-time environmental perception data and the historical environmental perception data; The monitoring module 350 is used to determine the safety monitoring results of the target cabin based on the cabin visual status information, the cabin audio event information, the environmental change rate, and the real-time access control data.

[0094] Based on the above embodiments, the real-time environmental sensing data includes real-time ambient light intensity, and the device further includes: The illumination determination module is used to determine whether the real-time ambient light intensity is less than a preset light intensity before performing image enhancement processing on the real-time image data to obtain the target image data; if the real-time ambient light intensity is less than the preset light intensity, then the real-time image data is subjected to image enhancement processing to obtain the target image data; if the real-time ambient light intensity is not less than the preset light intensity, then the real-time image data is determined as the target image data.

[0095] Based on the above embodiments, the monitoring module 350 is specifically used for: Based on the cabin visual status information, the cabin audio event information, and the environmental change rate, the general risk monitoring result of the target cabin is determined; based on the cabin visual status information, the cabin audio event information, and the real-time access control data, the intrusion risk monitoring result of the target cabin is determined; the general risk monitoring result and the intrusion risk monitoring result are integrated to obtain the security monitoring result of the target cabin.

[0096] Based on the above embodiments, the monitoring module 350 determines the intrusion risk monitoring results of the target cabin according to the cabin visual state information, the cabin audio event information, and the real-time access control data, including: Determine whether the real-time access control data indicates an abnormal door opening state; if the real-time access control data indicates an abnormal door opening state, determine the intrusion risk monitoring result based on the cabin visual state information and the cabin audio event information; if the real-time access control data does not indicate an abnormal door opening state, determine that the intrusion risk monitoring result indicates no intrusion risk.

[0097] Based on the above embodiments, the device further includes: The update module is used to determine whether the intrusion risk monitoring result indicates an intrusion risk after determining the intrusion risk monitoring result based on the cabin visual state information and the cabin audio event information; if the intrusion risk monitoring result indicates an intrusion risk, then the target image data is subjected to behavior recognition processing to obtain a behavior recognition result; and the intrusion risk monitoring result is updated based on the behavior recognition result to obtain an updated intrusion risk monitoring result.

[0098] Based on the above embodiments, the cabin visual state information includes visual type and recognition confidence level; the cabin audio event information includes audio event category and category confidence level; the monitoring module 350 determines the general risk monitoring results of the target cabin based on the cabin visual state information, the cabin audio event information, and the environmental change rate, including: Based on the visual type, the audio event category, the environmental change rate, and the risk feature library, candidate general risk types are determined; the environmental change rate is mapped to obtain an environmental change mapping value; the risk reliability of the candidate general risk type is calculated based on the recognition confidence level corresponding to the visual type, the category confidence level corresponding to the audio event category, and the environmental change mapping value; and the general risk monitoring result of the target cabin is determined based on the candidate general risk type and its corresponding risk reliability.

[0099] Based on the above embodiments, the device further includes: The sending module is used to send the safety monitoring results of the target cabin to the staff's terminal after determining the safety monitoring results of the target cabin based on the cabin visual state information, the cabin audio event information, the environmental change rate and the real-time access control data.

[0100] The safety monitoring device for unmanned vessel cabins provided in this embodiment of the invention can execute the safety monitoring method for unmanned vessel cabins provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0101] It is worth noting that in the embodiments of the above-mentioned safety monitoring device for unmanned ship cabins, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0102] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0103] like Figure 4 As shown, electronic device 4 is represented in the form of a general-purpose computing electronic device. The components of electronic device 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0104] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0105] Electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 4, including volatile and non-volatile media, removable and non-removable media.

[0106] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0107] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0108] Electronic device 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 4, and / or with any device that enables electronic device 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of electronic device 4 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be combined with electronic device 4 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0109] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, such as implementing the safety monitoring method for unmanned ship cabins provided in the embodiments of the present invention.

[0110] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the safety monitoring method for unmanned ship cabins provided in any embodiment of the present invention.

[0111] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements, for example, the safety monitoring method for unmanned ship cabins provided in this invention.

[0112] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0113] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0114] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0115] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0117] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with relevant laws and regulations.

[0118] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for safety monitoring of an unmanned vessel's cabin, characterized in that, The method includes: Acquire real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target compartment; The real-time image data is subjected to image enhancement processing to obtain target image data, and the real-time audio data is subjected to noise reduction processing to obtain target audio data; The target image data is subjected to target detection processing to obtain cabin visual state information; the target audio data is subjected to acoustic event recognition processing to obtain cabin audio event information. The rate of environmental change is calculated based on the real-time environmental perception data and the historical environmental perception data. The safety monitoring results of the target cabin are determined based on the cabin visual status information, cabin audio event information, environmental change rate, and real-time access control data.

2. The method according to claim 1, characterized in that, Real-time environmental perception data includes real-time ambient light intensity. Before performing image enhancement processing on the real-time image data to obtain the target image data, the data further includes: Determine whether the real-time ambient light intensity is less than the preset light intensity; If the real-time ambient light intensity is less than the preset light intensity, then the real-time image data is subjected to image enhancement processing to obtain the target image data; If the real-time ambient light intensity is not less than the preset light intensity, then the real-time image data is determined as the target image data.

3. The method according to claim 1, characterized in that, Based on the cabin visual state information, the cabin audio event information, the environmental change rate, and the real-time access control data, the security monitoring results of the target cabin are determined, including: Based on the cabin visual status information, the cabin audio event information, and the environmental change rate, the general risk monitoring results of the target cabin are determined; Based on the visual status information of the cabin, the audio event information of the cabin, and the real-time access control data, the intrusion risk monitoring results of the target cabin are determined; The general risk monitoring results and the intrusion risk monitoring results are integrated to obtain the security monitoring results of the target compartment.

4. The method according to claim 3, characterized in that, Based on the visual status information of the cabin, the audio event information of the cabin, and the real-time access control data, the intrusion risk monitoring results of the target cabin are determined, including: Determine whether the real-time access control data indicates an abnormal door opening status; If the real-time access control data indicates an abnormal door opening state, the intrusion risk monitoring result is determined based on the cabin visual state information and the cabin audio event information. If the real-time access control data does not indicate an abnormal door opening state, then the intrusion risk monitoring result is determined to be no intrusion risk.

5. The method according to claim 4, characterized in that, After determining the intrusion risk monitoring result based on the cabin visual state information and the cabin audio event information, the method further includes: Determine whether the intrusion risk monitoring result indicates the presence of an intrusion risk; If the intrusion risk monitoring result indicates the presence of an intrusion risk, then the target image data is subjected to behavior recognition processing to obtain a behavior recognition result; the intrusion risk monitoring result is updated based on the behavior recognition result to obtain an updated intrusion risk monitoring result.

6. The method according to claim 3, characterized in that, The cabin visual status information includes visual type and recognition confidence level; the cabin audio event information includes audio event category and category confidence level; Based on the cabin visual state information, the cabin audio event information, and the environmental change rate, the general risk monitoring results for the target cabin are determined, including: Based on the visual type, the audio event category, the environmental change rate, and the risk feature library, candidate general risk types are determined; The environmental change rate is mapped to obtain the environmental change mapping value; The risk reliability of the candidate general risk type is calculated based on the recognition confidence level corresponding to the visual type, the category confidence level corresponding to the audio event category, and the environmental change mapping value. Based on the candidate general risk types and their corresponding risk reliability, the general risk monitoring results for the target compartment are determined.

7. The method according to claim 1, characterized in that, After determining the security monitoring results of the target cabin based on the cabin visual state information, the cabin audio event information, the environmental change rate, and the real-time access control data, the process further includes: The safety monitoring results of the target compartment are sent to the staff's terminal.

8. A safety monitoring device for an unmanned ship cabin, characterized in that, The device includes: The acquisition module is used to acquire real-time image data, real-time audio data, real-time environmental perception data, and real-time access control data of the target compartment. The processing module is used to perform image enhancement processing on the real-time image data to obtain target image data, and to perform noise reduction processing on the real-time audio data to obtain target audio data; The recognition module is used to perform target detection processing on the target image data to obtain cabin visual state information; and to perform acoustic event recognition processing on the target audio data to obtain cabin audio event information. The calculation module is used to calculate the rate of environmental change based on the real-time environmental perception data and the historical environmental perception data; The monitoring module is used to determine the safety monitoring results of the target cabin based on the cabin visual status information, the cabin audio event information, the environmental change rate, and the real-time access control data.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the safety monitoring method for unmanned ship cabins as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the safety monitoring method for any of the unmanned vessel cabins as described in claims 1-7.

Images