A marine inspection robot control method and system

Abstract

The application relates to the technical field of robot control, and provides a marine inspection robot control method and system, the method of which comprises the following steps: acquiring individualized feature information corresponding to an instrument, and associating the individualized feature information with the corresponding instrument; acquiring an actual aging condition of the instrument, and updating the individualized feature information according to the actual aging condition; identifying the identity of a current instrument to be inspected; calling the individualized feature information from a pre-stored individualized feature information library; acquiring an image of the current instrument to be inspected; eliminating or weakening the interference of permanent physical defect information on reading identification according to the called individualized feature information; extracting a reading based on the processed image of the current instrument to be inspected; and judging whether the reading of the current instrument to be inspected is an abnormal reading according to the reading and the individualized feature information, so as to obtain a reading judgment result. The application has the effect of improving the automatic inspection efficiency and data reliability of the marine inspection robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a marine inspection robot control method and system. BACKGROUND

[0002] In modern industrial production and operation, especially in some areas that are not friendly to human work environment, such as high temperature, high humidity, strong vibration or closed space with potential danger, automated inspection robots are playing an increasingly important role. These robots are usually equipped with various sensors to replace manual routine inspection tasks, thereby improving efficiency and reducing risks. However, in actual application, due to the complexity of the environment and the inherent characteristics of the detected object, the data collected by the robot is often disturbed by various factors, which brings serious challenges to the accurate identification of data.

[0003] When the robot and the ship body sway, the robot body, the surrounding pipelines or other equipment will cast moving shadows on the glass cover of the instrument panel. Moreover, the light supplement lamp of the visual sensor itself or the external light source will form a high-intensity light reflection area on the smooth instrument glass cover at different observation angles. The position and shape of these light reflection points or light reflection bands change dramatically with slight changes in the observation angle, often blocking the key scale lines or pointers. In addition, these long-serving instruments are also aging. The glass cover of the instrument panel will become blurred due to the attachment of oil stains and dust, reducing the image contrast. The scale numbers and scale lines on the panel may fade or peel off due to long-term light and changes in temperature and humidity. The pointer itself may also appear rust, fade or slight deformation. These physical progressive deterioration further increases the difficulty of image recognition. Therefore, an instrument reading recognition method based on fixed template matching or simple feature extraction will significantly decrease in reliability when facing the above-mentioned complex scene with multiple interference factors superimposed. These single or combined disturbances result in frequent errors or failure to identify the recognition results, generating a large amount of invalid data and false alarms for the system, which greatly reduces the value of robot inspection automation and still requires a large amount of manual review to ensure the accuracy of the data.

[0004] In view of the above problems, the prior art needs to be improved. SUMMARY

[0005] The present application discloses a marine inspection robot control method and system, aiming to solve the problem that the marine inspection robot in a complex engine room environment has a decrease in instrument reading recognition accuracy due to the superposition of multiple interference factors such as ship body sway, light change and instrument aging, resulting in a large amount of invalid data and false alarms for the system.

[0006] The technical solution of the present application is as follows:

[0007] In a first aspect, the application discloses a ship inspection robot control method, which specifically comprises:

[0008] Before the inspection of the ship instrument, the individual characteristic information of the corresponding instrument is acquired, and the individual characteristic information is stored in association with the corresponding instrument; the individual characteristic information includes permanent physical defect information of the instrument panel and / or pointer;

[0009] The actual aging condition of the corresponding instrument is acquired, and the individual characteristic information is updated according to the actual aging condition;

[0010] In the inspection process of the ship inspection robot, the identity of the current instrument to be inspected is identified;

[0011] The individual characteristic information corresponding to the identity of the current instrument to be inspected is called from the pre-stored individual characteristic information library;

[0012] The image of the current instrument to be inspected is acquired;

[0013] According to the called individual characteristic information, the image of the current instrument to be inspected is processed to eliminate or weaken the interference of the permanent physical defect information on the reading identification;

[0014] Based on the processed image of the current instrument to be inspected, the reading of the current instrument to be inspected is extracted;

[0015] According to the reading of the current instrument to be inspected and the called individual characteristic information, it is judged whether the reading of the current instrument to be inspected is an abnormal reading, and a reading judgment result is obtained.

[0016] Through the technical scheme, the complex and changeable inspection environment in the ship engine room can be effectively coped with, the inherent physical defects and aging conditions of the instrument are preprocessed and compensated by introducing the individual characteristic information of the instrument and dynamically updating, the accuracy and robustness of the instrument reading identification under complex interference are significantly improved, thereby reducing the false alarm and missing report, and improving the inspection efficiency and data reliability.

[0017] Further, the application also provides a ship inspection robot control method, wherein the step of judging whether the reading of the current instrument to be inspected is an abnormal reading according to the reading of the current instrument to be inspected and the called individual characteristic information, and obtaining a reading judgment result comprises:

[0018] The reading of the current instrument to be inspected is acquired;

[0019] The individual characteristic information corresponding to the identity of the current instrument to be inspected is acquired;

[0020] The reading confidence of the current instrument to be inspected is calculated, and the reading confidence is subjected to threshold comparison to obtain a confidence comparison result;

[0021] When the confidence ratio result indicates that the confidence ratio is below a preset threshold, performing light mode switching on the image of the current instrument to be inspected;

[0022] During the light mode switching, a plurality of image frames of the current instrument to be inspected are continuously acquired;

[0023] The dynamic changes of the visual features of the suspected interference region in different light modes are compared;

[0024] According to the dynamic changes of the visual features, the interference region is determined and the corresponding interference property information is judged;

[0025] According to the interference property information, it is judged whether the reading of the current instrument to be inspected is an abnormal reading, and a reading judgment result is obtained.

[0026] Through the technical scheme, light mode switching and multi-frame image acquisition can be performed for low-confidence readings, and the interference property can be identified by analyzing the dynamic changes of the interference region under different light conditions, so that the reading abnormality can be more accurately judged, the real abnormality and environmental interference can be effectively distinguished, and misjudgment caused by single image quality problem can be avoided.

[0027] More specifically, in some embodiments, the step of judging whether the reading of the current instrument to be inspected is an abnormal reading according to the interference property information to obtain a reading judgment result comprises:

[0028] When the interference property information is instantaneous optical distortion and the overlap degree of the interference visual performance and the key reading region of the current instrument to be inspected reaches a preset value, the contribution weight of the pixel value of the interference region to the output of the reading recognition algorithm is calculated;

[0029] According to the contribution weight, the reading confidence of the current instrument to be inspected is adjusted;

[0030] If the adjusted reading confidence is still below the preset threshold, the reading of the current instrument to be inspected is re-extracted by weighted fusion of the features of the non-interference region to obtain a reading re-extraction result;

[0031] According to the interference property information and the reading re-extraction result, it is judged whether the reading of the current instrument to be inspected is an abnormal reading, and a reading judgment result is obtained.

[0032] Through the technical scheme, for the reading interference caused by instantaneous optical distortion, the reading confidence can be adjusted by evaluating the pixel contribution weight of the interference region, and when the confidence is still low, the features of the non-interference region can be used for weighted fusion to re-extract the reading, so that accurate reading can be obtained as much as possible under complex optical interference, and the adaptability of the system under harsh light conditions is improved.

[0033] On the basis of the above, the application further proposes that when the interference property information is instantaneous optical distortion and the degree of overlap between the interference visual performance and the current key reading area of the instrument to be inspected reaches a preset value, the step of calculating the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm comprises:

[0034] determining the overlapping range of the interference area and the current key reading area of the instrument to be inspected;

[0035] based on the overlapping range, evaluating the local image quality of the pixels in the overlapping area; the local image quality includes local contrast, edge definition, and brightness fluctuation amplitude;

[0036] according to the sensitivity mapping relationship between the interference property information and the reading recognition algorithm, adjusting the weight of the pixels in the overlapping area in the reading recognition algorithm;

[0037] dividing the overlapping area into sub-areas;

[0038] calculating the pixel contribution weight of each sub-area and performing weighted fusion to obtain the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm.

[0039] Through the technical solution, the local image quality of the interference area and the sensitivity to the reading recognition algorithm can be finely evaluated, and sub-area division and weighted fusion are performed, realizing pixel-level quantization of instantaneous optical distortion interference, so that the reading recognition algorithm can more intelligently avoid or compensate for interference, further improving the accuracy of reading recognition.

[0040] In some preferred embodiments, if the adjusted reading confidence is still lower than the preset threshold, the step of re-extracting the reading of the current instrument to be inspected by weighted fusion of the features of the non-interference area to obtain the reading re-extraction result comprises:

[0041] continuously capturing images of the current instrument to be inspected to obtain a preset number of images;

[0042] determining the possible distribution range of the corresponding interference according to the dynamic change characteristics of the instantaneous optical distortion area corresponding to the interference property information of instantaneous optical distortion;

[0043] real-time recognition of the non-interference area in the preset number of images;

[0044] adjusting the attention area for reading extraction according to the possible distribution range and the non-interference area in the preset number of images;

[0045] extracting local reading features from the non-interference area in the preset number of images and performing weighted fusion on the local reading features to re-extract the reading of the current instrument to be inspected to obtain the reading re-extraction result;

[0046] Evaluate the read confidence of the read re-extraction result.

[0047] By the technical solution, the undisturbed area can be identified in real time by continuously collecting images and combining the dynamic characteristics of instantaneous optical distortion, and the attention area of read extraction is adjusted, so that when part of the area is disturbed, the features of other clear areas can still be used for weighted fusion, thereby effectively improving the success rate and accuracy of read re-extraction.

[0048] Further, the step of evaluating the read confidence of the read re-extraction result comprises:

[0049] Identify the overlapping area of the permanent physical defect information in the instrument key read area and the instantaneous optical distortion;

[0050] Analyze the image feature degradation degree of different positions in the overlapping area; the degradation degree includes local contrast reduction, edge blur, and brightness fluctuation;

[0051] According to the degradation degree, determine the interference weight of each pixel in the overlapping area on read identification;

[0052] Combine the interference weight and the personalized feature information to perform compensatory processing on the read features in the overlapping area;

[0053] Based on the read features after compensatory processing, calculate the local confidence of the re-extracted read;

[0054] Weighted fusion is performed on the local confidence to obtain the overall confidence of the read re-extraction result, which is recorded as the read confidence of the read re-extraction result.

[0055] By the technical solution, the overlapping area of the permanent physical defect and the instantaneous optical distortion can be identified and analyzed, the image feature degradation degree can be quantified, and the compensatory processing can be performed in combination with the personalized feature information, so that the confidence of the re-extracted read can be more accurately evaluated, and the reliability of the read result under multiple disturbances is ensured.

[0056] On the basis described above, the step of weighted fusion of the local confidence to obtain the overall confidence of the read re-extraction result, which is recorded as the read confidence of the read re-extraction result, comprises:

[0057] Continuously monitor the image quality change trend of each local area in the instrument key read area;

[0058] According to the corresponding change rate and change amplitude in the image quality change trend, and combining the sensitivity of the image quality change to the read identification algorithm, dynamically adjust the weight of the local confidence;

[0059] The weight adjustment of the local confidence is periodically calibrated according to the frequency and duration of the current light mode switching and the possible distribution range of the instantaneous optical distortion;

[0060] The local confidences are weighted and fused based on the adjusted weight of the local confidence to obtain the overall confidence of the read value re-extraction result, which is recorded as the read value confidence of the read value re-extraction result.

[0061] Through the technical solution, the image quality change trend of each local area in the instrument key read value area can be continuously monitored, and the local confidence weight is dynamically adjusted according to the change rate, amplitude and algorithm sensitivity, and at the same time, the light mode switching and distortion range are periodically calibrated, so that the adaptive and high-precision evaluation of the read value confidence is realized, and the robustness of the system in a dynamic environment is further improved.

[0062] Preferably, the step of continuously monitoring the image quality change trend of each local area in the instrument key read value area comprises:

[0063] Starting the high-frequency image acquisition unit integrated on the marine inspection robot, continuously acquiring images of the instrument key read value area at a preset frequency to obtain high-frequency images;

[0064] The local image quality indicators in the high-frequency images are calculated in real time;

[0065] The local image quality indicators are processed using time series analysis method to smooth the instantaneous noise and highlight the real change trend of the image quality, so as to obtain the image quality change trend of each local area in the instrument key read value area.

[0066] Through the technical solution, the image quality change trend of the instrument key read value area can be monitored in real time and accurately through high-frequency image acquisition and time series analysis, and the instantaneous noise can be effectively filtered out, thereby providing a reliable data basis for subsequent confidence evaluation and interference processing.

[0067] In some embodiments, the step of continuously monitoring the image quality change trend of each local area in the instrument key read value area comprises:

[0068] When the instantaneous optical distortion appears in the high-frequency images, the specific spectrum light source carried by the marine inspection robot is controlled to irradiate the surface of the glass cover of the instrument to be inspected at a preset frequency and wavelength sequence;

[0069] The image frames of the instrument key read value area are acquired at different polarization angles using the polarized light imaging unit;

[0070] The response difference of the image features corresponding to each image frame under different spectrum bands is analyzed, and the polarization characteristic change of the interference area in the image under different polarization angles is analyzed;

[0071] According to the response difference and the polarization characteristic change, dynamic optical effects are identified and distinguished from visual features caused by inherent structures or background environments of the instrument;

[0072] If the identification result is salt crystals or oil film, the image quality evaluation weight of the corresponding area is adjusted according to the optical characteristic change trend of the salt crystals or oil film, and the glass cover cleaning module is triggered for processing.

[0073] By the technical solution, through the multispectral and polarized light imaging technology combined with dynamic analysis, transient optical distortion and inherent features of the instrument can be accurately identified and distinguished, especially salt crystals or oil film and other specific pollution can be identified, and the cleaning module is triggered, so that part of the interference source is fundamentally solved, and the automation and intelligent level of the inspection are significantly improved.

[0074] In a second aspect, the application also discloses a marine inspection robot control system for performing marine inspection robot control, specifically comprising:

[0075] A personal feature acquisition module is configured to acquire personal feature information of the corresponding instrument before the marine instrument is inspected, and store the personal feature information in association with the corresponding instrument; the personal feature information includes permanent physical defect information of the instrument panel and / or the pointer;

[0076] An aging condition acquisition module is configured to acquire an actual aging condition of the corresponding instrument, and update the personal feature information according to the actual aging condition;

[0077] An instrument identity recognition module is configured to recognize the identity of the current instrument to be inspected during the inspection process of the marine inspection robot;

[0078] A personal feature retrieval module is configured to retrieve personal feature information corresponding to the identity of the current instrument to be inspected from a pre-stored personal feature information database;

[0079] An instrument image acquisition module is configured to acquire an image of the current instrument to be inspected;

[0080] An interference elimination module is configured to process the image of the current instrument to be inspected according to the retrieved personal feature information, so as to eliminate or weaken the interference of the permanent physical defect information on the reading recognition;

[0081] An instrument reading extraction module is configured to extract the reading of the current instrument to be inspected based on the processed image of the current instrument to be inspected;

[0082] A reading result judgment module is configured to judge whether the reading of the current instrument to be inspected is an abnormal reading according to the reading of the current instrument to be inspected and the retrieved personal feature information, and obtain a reading judgment result.

[0083] The technical solution provides a system-level solution. Through modular design, it enables the acquisition, storage, updating, and retrieval of personalized instrument features, as well as image processing and reading judgment based on these features. This effectively solves the problem of accuracy in instrument reading recognition under complex environments and improves the automation and intelligence level of marine inspection robots.

[0084] Beneficial effects

[0085] The control method for a marine inspection robot disclosed in this application effectively eliminates the interference of inherent defects such as aging and wear of the instruments on reading recognition by acquiring and associating personalized feature information (including information on permanent physical defects) of the instruments before inspection and preprocessing the images based on this information during the inspection process. Furthermore, this method can dynamically update the personalized feature information according to the actual aging condition of the instruments, enabling the system to adapt to long-term changes in the instruments. After reading recognition, the personalized feature information is used to determine whether the reading is abnormal, further improving the accuracy of the judgment. Compared with existing methods that simply rely on fixed templates or feature extraction, this application can effectively address the problem of inaccurate data caused by the superposition of multiple complex interferences in the ship's engine room, such as hull movement, changes in lighting, and instrument aging. It significantly improves the accuracy and robustness of instrument reading recognition, reduces invalid data and erroneous alarms, and thus greatly improves the automated inspection efficiency and data reliability of the marine inspection robot. Attached Figure Description

[0086] Figure 1 This is a flowchart of a method for controlling a marine inspection robot in one embodiment of the present invention;

[0087] Figure 2 This is a flowchart of a method for controlling a marine inspection robot according to another embodiment of the present invention;

[0088] Figure 3 This is a system block diagram of a marine inspection robot control system according to another embodiment of the present invention;

[0089] Explanation of reference numerals in the attached figures:

[0090] 1. Marine inspection robot control system; 11. Individual feature acquisition module; 12. Aging status acquisition module; 13. Instrument identification module; 14. Individual feature retrieval module; 15. Instrument image acquisition module; 16. Interference recognition cancellation module; 17. Instrument reading extraction module; 18. Reading result judgment module. Detailed Implementation

[0091] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0092] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0093] Traditional marine inspection robots often encounter various interferences when inspecting ship instruments due to the complexity of the engine room environment, such as high temperature, high humidity, strong vibration, complex electromagnetic interference, and ship rolling. These interferences include perspective distortion, moving shadows, high-intensity reflections, and instrument aging (such as blurred glass covers, faded dials, and corroded pointers). These interferences significantly reduce the reliability of instrument reading recognition methods based on fixed template matching or simple feature extraction, leading to frequent errors or failures in recognition. This results in a large amount of invalid data and false alarms, greatly diminishing the automation value of robot inspections and requiring extensive manual verification to ensure data accuracy. Failure to address these issues will severely impact the automation level and data accuracy of marine inspection robots, increase manual verification costs, and potentially lead to safety hazards due to misreading critical instrument data.

[0094] In response, this application proposes a control method for a marine inspection robot, combining... Figure 1 As shown, it includes:

[0095] S1. Before inspecting the ship's instruments, obtain the personalized feature information of the corresponding instruments and associate and store the personalized feature information with the corresponding instruments; the personalized feature information includes information on permanent physical defects of the instrument panel and / or pointers.

[0096] S2, obtain the actual aging status of the corresponding instrument, and update the personalized feature information according to the actual aging status;

[0097] S3 identifies the identity of the instrument to be inspected during the inspection process of the marine inspection robot.

[0098] S4, retrieve the personalized feature information corresponding to the identity of the instrument to be inspected from the pre-stored personalized feature information database;

[0099] S5, acquire the image of the instrument currently to be inspected;

[0100] S6, based on the retrieved personalized feature information, processes the image of the instrument to be inspected to eliminate or reduce the interference of permanent physical defects on reading recognition;

[0101] S7, Based on the processed image of the instrument to be inspected, extract the reading of the instrument to be inspected.

[0102] S8, based on the current reading of the instrument to be inspected and the retrieved personalized feature information, determines whether the current reading of the instrument to be inspected is an abnormal reading, and obtains the reading judgment result.

[0103] Before inspecting ship instruments, it is necessary to obtain the unique characteristics of each instrument and associate this information with the corresponding instrument. Unique characteristics can include information on permanent physical defects on the instrument panel and / or pointers. For example, this can be done manually by operators visually inspecting each instrument during the initial robot deployment, recording scratches, stains, and faded areas on the panel, or rust and bending on the pointers, and then associating this information with the corresponding instrument number. Alternatively, a high-resolution camera can be used to acquire initial images of the instruments, and image analysis software can automatically identify and extract this information. For example, edge detection and texture analysis algorithms can be used to identify scratches and stains, and color analysis can be used to identify faded areas. This information can then be stored in a database as structured data and associated with the corresponding instrument identity (such as instrument number and installation location).

[0104] Furthermore, it is necessary to obtain the actual aging condition of the corresponding instrument and update the personalized feature information accordingly. For example, regular manual inspections can be used to record the degree of blurring of the instrument glass, the wear of the dial markings, and the degree of corrosion of the pointers, and these aging conditions can be added to the database as new personalized feature information. Alternatively, sensors mounted on robots can be used, such as image analysis algorithms, to periodically evaluate the light transmittance of the instrument glass, the clarity of the dial markings, and the integrity of the pointers, and these evaluation results can be used as the actual aging condition. Based on these conditions, the stored personalized feature information can be automatically updated or corrected. For example, if the blurring of the glass is detected to be worsening, a "glass glass blurring" mark can be added to the personalized feature information, and its degree can be recorded.

[0105] During the inspection process of marine inspection robots, it is necessary to identify the identity of the instruments to be inspected. For example, QR codes or RFID tags can be placed near each instrument, and the robot can identify the instrument by scanning these tags. Alternatively, image recognition technology can be used, employing a pre-trained deep learning model to identify the appearance features of the instruments (such as shape, color, brand logo, etc.) to determine the identity of the instrument to be inspected.

[0106] The system retrieves personalized feature information corresponding to the identity of the instrument to be inspected from a pre-stored personalized feature information database. For example, after identifying the identity of the instrument to be inspected, the system will query and extract the personalized feature information associated with the instrument from the database based on the identity information, including its permanent physical defects and the latest aging status.

[0107] Acquire images of the instruments currently under inspection. For example, when a marine inspection robot is inspecting an instrument, its industrial camera will take one or more images of the instrument. These images can be visible light images or other spectral images.

[0108] Based on the retrieved personalized feature information, the image of the instrument to be inspected is processed to eliminate or reduce the interference of permanent physical defects on reading recognition. For example, if the personalized feature information indicates scratches on the instrument panel, image restoration algorithms can be used to fill pixels or synthesize textures in the scratched areas of the image to eliminate the interference of scratches on reading recognition. If the personalized feature information indicates stains on the instrument glass cover, image enhancement algorithms, such as contrast stretching and brightness adjustment, can be used to reduce the impact of stains on image clarity.

[0109] Based on the processed image of the instrument to be inspected, the reading of the instrument is extracted. For example, traditional image processing algorithms, such as Hough transform to detect the pointer and circular detection to identify the instrument panel, can be used, and then the reading can be calculated based on the relative position of the pointer and the scale lines. Alternatively, deep learning models, such as convolutional neural networks (CNNs), can be used to directly identify the instrument panel, scale lines, and pointer from the processed image and output the reading.

[0110] Based on the current reading of the instrument to be inspected and the retrieved personalized feature information, it is determined whether the current reading of the instrument to be inspected is abnormal, and a reading judgment result is obtained. For example, the retrieved reading can be compared with a preset normal reading range; if it exceeds the range, it is judged as abnormal. At the same time, personalized feature information is combined; for example, if the personalized feature information indicates that the instrument has serious aging, which may cause reading deviation, this deviation will be taken into account when judging abnormalities to avoid false alarms.

[0111] Specifically, before inspecting ship instruments, this application first acquires and stores the personalized characteristic information of the instruments, including information on permanent physical defects of the instrument panel and / or pointers. This differs from existing methods that rely solely on general templates for identification, as these methods cannot effectively address the identification challenges caused by individual instrument differences and long-term wear. By pre-knowing these unique physical defects, this application can provide a customized basis for subsequent image processing. For example, when a specific scratch exists on the instrument panel, traditional methods may misidentify it as part of a scale line or pointer, leading to incorrect readings; however, this application can identify the scratch as a known defect and target its elimination or reduction during the image processing stage.

[0112] Furthermore, this application also acquires and updates personalized feature information based on the actual aging condition of the instrument. This means that the method can dynamically adapt to changes in the instrument over time, such as blurring of the glass cover or fading of the scale. Traditional methods often ignore the gradual aging of the instrument, causing the recognition performance to decline as the instrument ages. This application ensures that the recognition algorithm is always optimized based on the latest instrument condition by continuously updating the aging information, thereby maintaining high accuracy.

[0113] During the inspection process of the marine inspection robot, this application first identifies the identity of the instrument to be inspected and retrieves the corresponding personalized feature information from a pre-stored personalized feature information database. This step ensures that subsequent image processing and reading judgment are tailored to the specific instrument, rather than using a general strategy that applies to all instruments.

[0114] Subsequently, images of the instruments to be inspected are acquired, and the images are processed based on retrieved personalized feature information to eliminate or reduce interference from permanent physical defects on reading recognition. For example, if personalized feature information indicates a stain in a specific location on the instrument's glass cover, the system can apply local image enhancement or repair algorithms to process only that stained area, avoiding unnecessary modifications to other normal areas. This precise processing based on personalized information, compared to the blind application of general image processing algorithms in traditional methods, can more effectively remove interference while retaining key reading information, significantly improving image clarity and recognition usability.

[0115] Based on the processed image, the readings of the instruments currently under inspection are extracted. Because the image has been optimized to eliminate interference from permanent physical defects, the accuracy of reading extraction is significantly improved.

[0116] Finally, based on the readings of the instruments to be inspected and the retrieved personalized feature information, it is determined whether the readings are abnormal. This step not only considers whether the readings themselves exceed the preset range, but also incorporates the personalized features of the instruments. For example, if an instrument's pointer is slightly bent due to long-term aging, its readings may have a systematic deviation. This application can compensate for this deviation based on personalized feature information to avoid false alarms. This comprehensive judgment mechanism makes the identification of abnormal readings more intelligent and accurate, reducing false alarms and missed alarms.

[0117] Optional, combined Figure 2 As shown, S8 determines whether the reading of the instrument to be inspected is abnormal based on the current reading and the retrieved personalized feature information, and obtains the reading judgment result by including the following steps:

[0118] S81, obtain the reading of the instrument to be inspected;

[0119] S82, obtain personalized feature information corresponding to the identity of the instrument to be inspected;

[0120] S83, calculate the confidence level of the reading of the instrument to be inspected, and perform threshold comparison on the confidence level of the reading to obtain the confidence level comparison result;

[0121] S84, when the confidence comparison result is lower than the preset threshold, switch the illumination mode of the image of the instrument to be inspected.

[0122] S85 continuously acquires several image frames of the instrument to be inspected during the lighting mode switching process.

[0123] S86, compare the dynamic changes of visual features of suspected interference areas in all image frames under different lighting modes;

[0124] S87, based on the dynamic changes corresponding to visual features, determine the interference area and identify the corresponding interference nature information;

[0125] S88 determines whether the reading of the instrument to be inspected is abnormal based on the interference information, and obtains the reading judgment result.

[0126] Specifically, after obtaining the reading of the instrument to be inspected and the personalized feature information corresponding to its identity, the confidence level of the reading needs to be calculated. The confidence level can be understood as a quantitative assessment of the reliability of the extracted reading. For example, it can be comprehensively scored using image recognition algorithms based on multiple dimensions such as the clarity, completeness, and matching degree with known digit templates of the reading area. This confidence level is then compared with a preset threshold to determine whether the current reading is sufficiently reliable. For example, if the confidence level is lower than the preset threshold, it indicates that the current reading may be affected by interference or the image quality is poor, requiring further verification.

[0127] When the confidence level comparison result of the reading is lower than a preset threshold, in order to more accurately obtain instrument readings and identify potential interference, the marine inspection robot can switch the lighting mode of the image of the instrument to be inspected. The lighting mode switching may include adjusting the brightness, color, angle of the light source, or turning auxiliary light sources on / off. The purpose is to observe the response of suspected interference areas in the image by changing the lighting conditions, thereby distinguishing between true reading characteristics and transient interference.

[0128] During the switching of lighting modes, the marine inspection robot continuously acquires several image frames of the instrument to be inspected. These image frames record the visual information of the instrument panel under different lighting conditions, providing dynamic data for subsequent interference analysis.

[0129] The system then compares the dynamic changes of visual features of suspected interference areas across all image frames under different lighting conditions. For example, features of a genuine instrument reading area (such as pointers and scale lines) typically exhibit stable geometry and relatively consistent brightness / contrast changes under varying lighting conditions, while transient optical distortions (such as reflections and shadows) may show rapid changes in brightness, shape, or position. By analyzing these dynamic changes, permanent physical defects, genuine reading features, and transient interference can be effectively distinguished.

[0130] Based on the dynamic changes corresponding to visual features, the system can identify interference areas and determine the corresponding interference characteristics. For example, if an area exhibits rapid flickering or shape distortion during lighting mode switching, it may be identified as transient optical distortion; if an area remains blurry or missing under different lighting conditions, it may be related to permanent physical defects. Interference characteristics can include transient optical distortion, localized shadows, stains, etc.

[0131] Ultimately, the system determines whether the reading of the instrument under inspection is abnormal based on the interference characteristics, thus obtaining a reading judgment result. This means that even if the initial reading has a low confidence level, the system will not immediately classify it as abnormal, but will conduct a more comprehensive assessment based on the judgment of the interference characteristics. For example, if the low confidence level is caused by transient reflection, and the reflected area does not affect the critical reading, it may still be judged as a normal reading; conversely, if the interference characteristics indicate that the reading area is severely obstructed or distorted, it may be judged as abnormal.

[0132] In some preferred embodiments, a specific example is given below. Suppose a marine inspection robot is inspecting a pressure gauge. After initial image acquisition, the gauge reading is identified as "5.0 MPa," but its confidence level is calculated to be 0.6, lower than the preset threshold of 0.8. This indicates that the current reading may be uncertain. Instead of immediately determining whether the reading is abnormal, the system triggers a lighting mode switch. For example, the robot can turn on its auxiliary LED light source and adjust its illumination angle, while continuously acquiring five frames of images. Comparing these five frames, the system finds an area on the gauge glass that exhibits significant brightness flickering and shape changes under different lighting angles, while the gauge pointer and scale remain relatively stable. By analyzing this dynamic change, the system determines that the flickering area is a transient reflection, belonging to transient optical distortion. Since the reflective area does not completely obscure or distort the critical reading area, the system can reassess the reliability of the reading "5.0 MPa" based on this interference information. For example, if the reflection only affects the edge of the reading area while the core reading characteristics are clear, the system can determine that the reading is still valid. Combined with a preset abnormal range (e.g., the normal range for a pressure gauge is 3.0-4.5 MPa), the system ultimately determines that "5.0 MPa" is an abnormal reading. Conversely, if the reflection severely obscures the reading, the system may further trigger a reading re-extraction process, or, if it cannot accurately identify the reading, mark it as requiring manual verification. In this way, this application avoids simply misjudging low-confidence readings caused by momentary reflection as "unidentifiable" or "abnormal," instead performing a deeper analysis and verification, thereby improving the accuracy of the judgment.

[0133] Optionally, based on the interference characteristics information, the steps to determine whether the current reading of the instrument to be inspected is an abnormal reading, and to obtain the reading determination result, include:

[0134] When the interference information is instantaneous optical distortion and the degree of overlap between the interference visual performance and the key reading area of ​​the instrument to be inspected reaches a preset value, the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm is calculated.

[0135] Adjust the confidence level of the readings of the instruments to be inspected based on their contribution weights;

[0136] If the adjusted reading confidence level is still lower than the preset threshold, the reading of the instrument to be inspected is re-extracted by weighted fusion of the features of the undisturbed area, and the reading re-extraction result is obtained.

[0137] Based on the interference characteristics and the reading re-extraction results, determine whether the reading of the instrument to be inspected is an abnormal reading, and obtain the reading judgment result.

[0138] Specifically, transient optical distortion can be understood as an image distortion phenomenon that appears and may disappear within a short period of time, caused by external environmental factors (such as changes in illumination, reflection, glare, transient water mist, or oil stains). The key reading area of ​​an instrument refers to the core area on the instrument panel that directly displays the reading, such as pointer dials or digital displays. When the visual manifestation of transient optical distortion overlaps with the key reading area of ​​the instrument to a preset value, it means that the distortion has a significant potential impact on reading recognition, requiring further analysis. At this point, it is necessary to calculate the contribution weight of the pixel values ​​in the interference area to the output of the reading recognition algorithm. This contribution weight aims to quantify the degree of influence of the pixels in the interference area on the final reading recognition result. For example, by analyzing the brightness, contrast, and texture features of the interference area, combined with the sensitivity of the reading recognition algorithm to these features, the intensity of its interference with reading recognition can be evaluated.

[0139] The confidence level of the current instrument reading can be adjusted based on its contribution weight. For example, if the contribution weight of the interference area is high, indicating a significant impact on the reading identification result, the confidence level of the current reading can be reduced accordingly to reflect the uncertainty of the reading. Conversely, if the contribution weight is low, the confidence level of the reading may remain at a high level.

[0140] In practical applications, if the adjusted reading confidence level is still lower than the preset threshold, it indicates that even considering the influence of interference, the reliability of the current reading is still insufficient. To improve the accuracy of reading judgment, this application proposes to re-extract the reading of the instrument under inspection by weighted fusion of features from undisturbed areas, resulting in a re-extraction result. Undisturbed areas refer to parts of the image that are not significantly affected by transient optical distortion; these areas have relatively high image quality and can provide more reliable reading information. Weighted fusion can be understood as combining local reading features from different undisturbed areas and assigning different weights based on their reliability, thereby obtaining a more accurate and robust reading.

[0141] Therefore, based on the information about the nature of the interference and the results of the reading re-extraction, a final judgment can be made as to whether the reading of the instrument to be inspected is abnormal, thus obtaining the reading judgment result. This step combines a deep understanding of the nature of the interference with the reading re-extracted from the reliable area, making the judgment of abnormal readings more accurate.

[0142] Optionally, when the interference information is instantaneous optical distortion and the degree of overlap between the visual appearance of the interference and the key reading area of ​​the instrument to be inspected reaches a preset value, the step of calculating the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm includes:

[0143] Determine the overlap range between the interference area and the critical reading area of ​​the instrument to be inspected;

[0144] Based on the overlap range, the local image quality of pixels within the overlap region is evaluated; local image quality includes local contrast, edge sharpness, and brightness fluctuation.

[0145] Based on the mapping relationship between interference characteristics and the sensitivity of the reading recognition algorithm, the weights of pixels in the overlapping area in the reading recognition algorithm are adjusted.

[0146] Divide the overlapping regions into sub-regions;

[0147] The pixel contribution weights of each sub-region are calculated and weighted fusion is performed to obtain the contribution weights of the pixel values ​​of the interference region to the output of the reading recognition algorithm.

[0148] Determining the overlap between the interference area and the critical reading area of ​​the instrument to be inspected can be achieved using image segmentation techniques or geometric analysis-based methods. For example, by binarizing the image, the outline of the interference area can be identified, and its intersection with the predefined boundary of the critical reading area of ​​the instrument can be calculated to accurately determine the set of overlapping pixels.

[0149] Furthermore, based on the overlap range, the local image quality of pixels within the overlapping region is evaluated. Evaluation metrics for local image quality include local contrast, edge sharpness, and brightness fluctuation. Local contrast can be obtained by calculating the root mean square contrast (RMS contrast) of pixel grayscale values ​​within the overlapping region, or by measuring the difference between the maximum and minimum pixel grayscale values. Edge sharpness can be achieved by processing the overlapping region using edge detection operators such as Sobel, Prewitt, or Canny, and calculating the average gradient magnitude of the edge response or the density of zero-crossing points. Brightness fluctuation can be characterized by calculating the standard deviation of pixel brightness within the overlapping region to reflect the impact of uneven illumination or transient flicker. A comprehensive evaluation of these metrics quantifies the specific impact of transient optical distortion on image quality.

[0150] Furthermore, based on the mapping relationship between interference characteristics and the sensitivity of the reading recognition algorithm, the weights of pixels within the overlapping region are adjusted. This sensitivity mapping relationship can be pre-trained using extensive experimental data. For example, the impact of different types of transient optical distortions (such as reflections, glare, and localized shadows) on the performance of specific reading recognition algorithms (such as OCR and pointer recognition) can be analyzed. When specific interference characteristics are identified, the corresponding weight adjustment factor can be found in this mapping relationship and applied to pixels within the overlapping region to reduce or increase the contribution of these pixels in the reading recognition process, thereby mitigating the impact of interference.

[0151] To achieve more refined evaluation and processing, the overlapping region is subdivided. This subdivision can be based on a fixed grid, adaptive quadtree decomposition, or clustering methods based on image features. For example, the overlapping region can be divided into several equal-sized sub-blocks, or dynamically divided based on local differences in image quality metrics, resulting in relatively uniform image quality within each sub-region.

[0152] Finally, the pixel contribution weights of each sub-region are calculated and weighted fused to obtain the contribution weight of the pixel values ​​of the interference region to the output of the readout recognition algorithm. The pixel contribution weight of each sub-region can be calculated independently based on its local image quality assessment results, interference nature, and sensitivity mapping relationship with the readout recognition algorithm. Subsequently, the contribution weights of these sub-regions can be weighted and averaged or accumulated according to their importance in the entire overlapping region (e.g., proximity to key readout features, area size, etc.) to obtain a comprehensive contribution weight that accurately reflects the impact of instantaneous optical distortion on the readout recognition algorithm.

[0153] Optionally, if the adjusted reading confidence level is still lower than the preset threshold, the reading of the instrument to be inspected is re-extracted by weighted fusion of features from the undisturbed area. The steps to obtain the reading re-extraction result include:

[0154] The system continuously acquires images of the instruments to be inspected, resulting in a preset number of frames.

[0155] Based on the dynamic change characteristics of the instantaneous optical distortion region corresponding to the interference nature information, the possible distribution range of the corresponding interference is determined;

[0156] Real-time identification of undisturbed areas in images with a preset number of frames;

[0157] Adjust the area of ​​interest for reading extraction based on the possible distribution range and the undisturbed area in the preset frame number of images;

[0158] Local reading features are extracted from undisturbed areas in images with a preset number of frames, and weighted fusion is performed on the local reading features to re-extract the readings of the instruments to be inspected, resulting in the reading re-extraction result.

[0159] Evaluate the confidence level of the readings obtained from the reading re-extraction results.

[0160] Specifically, continuously acquiring images of the instruments to be inspected to obtain a preset number of frames refers to using image acquisition equipment mounted on a marine inspection robot to continuously capture video streams or a series of consecutive image frames of the area to be inspected at a certain frame rate. The preset number of frames can be understood as a sequence of images acquired within a specific time window, sufficient to capture the dynamic changes in instantaneous optical distortion. Its purpose is to provide sufficient temporal information for subsequent interference analysis and identification of undisturbed areas.

[0161] Specifically, determining the possible distribution range of transient optical distortion based on its dynamic characteristics in the region corresponding to the transient optical distortion refers to predicting or estimating the potential location and impact range of these disturbances on the dashboard surface based on the analysis of the dynamic features (such as flickering, movement, and shape changes) exhibited by transient optical distortions (e.g., reflections, glare, water droplets, oil stains, etc.) in the image. This can be achieved through pre-trained models or real-time analysis of drastic changes in pixel values ​​and light spot movement trajectories in image sequences. The purpose is to initially delineate the areas where interference may exist, providing guidance for accurately identifying undisturbed areas.

[0162] In practical applications, real-time identification of undisturbed areas within a preset number of image frames involves using image processing and computer vision algorithms to analyze continuously acquired image frames frame by frame. This process eliminates known permanent physical defects and dynamically changing instantaneous optical distortion areas, thereby accurately identifying dashboard areas that are not significantly disturbed at the current moment. For example, background modeling, motion detection, optical flow analysis, or deep learning models can be used to distinguish stable areas from dynamically disturbed areas. The goal is to ensure that the image areas relied upon for subsequent reading extraction are clear and reliable.

[0163] Furthermore, adjusting the focus area for reading extraction based on the possible distribution range of interference and the undisturbed areas in the preset number of frames means that after determining the possible distribution range of interference and the actual undisturbed areas, the system dynamically adjusts the focus of the reading recognition algorithm, making it primarily focus on those areas confirmed to be undisturbed and containing key reading information. This can be manifested by cropping the image, masking it, or adjusting the attention weights in the neural network, with the aim of avoiding the negative impact of interference areas on reading recognition and improving recognition efficiency and accuracy.

[0164] Therefore, local reading features are extracted from undisturbed areas in a preset number of image frames, and these local reading features are weighted and fused to re-extract the reading of the instrument to be inspected, resulting in a reading re-extraction result. This refers to extracting local features such as numbers, scales, and pointer positions on the dashboard surface within the adjusted area of ​​interest using image recognition algorithms (such as OCR, pointer recognition, etc.). Since these local features come from multiple undisturbed image frames or regions, they can be combined using weighted averaging, majority voting, or confidence-based fusion algorithms to form a more stable and accurate reading result. The aim is to overcome the minor interference or recognition errors that may exist in a single image frame or local region through multi-source information fusion, thereby improving the robustness of reading re-extraction.

[0165] Finally, evaluating the confidence level of the re-extracted readings refers to assessing the reliability of the re-extracted readings. This may include calculating the matching degree between the reading and known scale lines, the probability score output by the recognition algorithm, or the consistency with readings in the same area in other frames. The purpose is to provide a quantitative basis for reliability in subsequent anomaly detection, ensuring that the re-extracted readings are trustworthy.

[0166] Optionally, the steps for evaluating the confidence level of the readings retrieved include:

[0167] Identify the overlapping areas of permanent physical defects and transient optical distortions within the key reading areas of the instrument;

[0168] Analyze the degree of image feature degradation at different locations within the overlapping region; the degree of degradation includes reduced local contrast, blurred edges, and brightness fluctuations;

[0169] Based on the degree of degradation, determine the interference weight of each pixel in the overlapping area for reading recognition;

[0170] By combining interference weights and personalized feature information, compensatory processing is performed on the reading features in the overlapping area;

[0171] Based on the reading characteristics after compensatory processing, the local confidence level of the re-extracted readings is calculated;

[0172] The local confidence scores are weighted and fused to obtain the overall confidence score of the reading re-extraction result, which is recorded as the reading confidence score of the reading re-extraction result.

[0173] Specifically, identifying the overlapping area between permanent physical defects and transient optical distortions within the critical reading area of ​​an instrument involves using techniques such as image registration, feature matching, and region segmentation to spatially align and overlay pre-stored personalized feature information (including information on permanent physical defects) with the transient optical distortion areas identified in the currently acquired instrument image. This allows for precise determination of the common coverage area of ​​both within the critical reading area. This overlapping area is the most susceptible to interference and the most complex region in reading recognition.

[0174] Analyzing the degree of image feature degradation at different locations within the overlapping region can be understood as performing a refined analysis of the image data within the overlapping region to quantify the degree of visual quality decline. The assessment of degradation can include: reduced local contrast, which measures the sharpness of image details by calculating the variance or standard deviation of local pixel grayscale values; blurred edges, which assesses the sharpness of image edges by analyzing image gradient information or using edge detection algorithms such as the Laplacian operator; and brightness fluctuations, which reflect the uniformity of illumination by statistically analyzing the brightness distribution of local pixels or calculating the mean and standard deviation of brightness. These indicators can comprehensively reflect the image quality within the overlapping region.

[0175] In practical applications, the interference weight of each pixel in the overlapping area is determined based on the degree of degradation. Specifically, this involves establishing a mapping relationship or using a machine learning model based on degradation indicators such as reduced local contrast, blurred edges, and brightness fluctuations obtained from the above analysis, to assign a weight value to each pixel in the overlapping area. This weight value reflects the degree of interference that pixel may introduce during the reading recognition process; the higher the degree of degradation, the greater the interference weight.

[0176] Furthermore, by combining interference weights and personalized feature information, compensatory processing is applied to the reading features within the overlapping region. This means that when extracting reading features, not only the visual information of the current image is considered, but also pre-acquired personalized feature information (such as information on permanent physical defects in the dashboard and / or pointers) is used to distinguish between permanent defects and transient interference. By applying interference weights to pixels or features within the overlapping region, damaged reading features can be enhanced, repaired, or denoised. For example, for local contrast reduction caused by permanent scratches, local contrast enhancement can be performed; for brightness fluctuations caused by transient reflections, local brightness correction can be performed, thereby restoring the original, undisturbed feature representation as much as possible.

[0177] Therefore, calculating the local confidence score of the re-extracted readings based on the compensated reading features involves analyzing these processed features using a reading recognition algorithm after compensating for the reading features within the overlapping region, and calculating a confidence score for each local region or each reading component. This local confidence score reflects the reliability of the reading recognition result in that local region after considering and compensating for interference factors.

[0178] Finally, the local confidence scores are weighted and fused to obtain the overall confidence score of the reading re-extraction result, which is recorded as the reading confidence score of the reading re-extraction result. This means that the confidence scores of each local region are weighted and averaged or other fusion algorithms are used according to their importance or degree of interference in the overall reading to obtain a comprehensive overall confidence score that represents the reliability of the entire reading re-extraction result. This overall confidence score will be used as the final confidence score of the reading re-extraction result for subsequent abnormal reading judgment.

[0179] Optionally, the step of weighted fusion of local confidence scores to obtain the overall confidence score of the reading re-extraction result, and recording it as the reading confidence score of the reading re-extraction result, includes:

[0180] Continuously monitor the image quality change trend of each local area within the key reading area of ​​the instrument;

[0181] Based on the rate and magnitude of change in the image quality change trend, and combined with the sensitivity of the image quality change to the reading recognition algorithm, the weight of the local confidence score is dynamically adjusted.

[0182] Based on the frequency and duration of the current illumination mode switching, as well as the possible distribution range of transient optical distortion, the weight adjustment of the local confidence level is periodically calibrated.

[0183] Based on the adjusted weights of the local confidence scores, the local confidence scores are weighted and fused to obtain the overall confidence score of the reading re-extraction result, which is recorded as the reading confidence score of the reading re-extraction result.

[0184] Specifically, continuously monitoring the image quality change trends of various local areas within the key reading areas of instruments refers to continuously and frequently capturing images of the key reading areas of instruments using image acquisition equipment integrated on a marine inspection robot, and analyzing the local quality indicators of these images in real time, such as local contrast, sharpness, and noise level. Time series analysis methods can be used to smooth out instantaneous noise, thereby revealing the true patterns and trends of image quality changes.

[0185] Specifically, based on the rate and magnitude of change in image quality trends, and considering the sensitivity of image quality changes to the reading recognition algorithm, the weight of local confidence is dynamically adjusted. This can be understood as follows: when the image quality of a certain local area significantly decreases or fluctuates, the weight of its corresponding local confidence in the overall confidence calculation should be reduced accordingly; conversely, when the image quality is stable or improves, its weight can be appropriately increased. This adjustment also needs to consider the sensitivity of the reading recognition algorithm to different degrees of image quality degradation. For example, some algorithms may be more sensitive to edge blurring, while others may be more sensitive to brightness fluctuations.

[0186] In practical applications, the weights of local confidence are periodically calibrated based on the frequency and duration of current illumination mode switching, as well as the possible distribution range of transient optical distortions. The aim is to further improve the accuracy of these weight adjustments. For example, in environments with frequent illumination mode switching, transient optical distortions may occur more frequently, and their distribution range may vary with the illumination angle. In such cases, it is necessary to periodically review and revise the previously determined weight adjustment strategy based on this dynamic information to ensure that the weight adjustments can adapt to real-time environmental changes.

[0187] Optionally, in the step of weighted fusion of local confidence scores to obtain the overall confidence score of the reading re-extraction result and recording it as the reading confidence score of the reading re-extraction result, the step of continuously monitoring the image quality change trend of each local area within the key reading area of ​​the instrument may include the following:

[0188] The high-frequency image acquisition unit integrated on the marine inspection robot is activated to continuously acquire images of key instrument reading areas at a preset frequency, thus obtaining high-frequency images.

[0189] Real-time calculation of local image quality indicators in high-frequency images;

[0190] By using time series analysis, local image quality indicators are processed to smooth out instantaneous noise and highlight the true trend of image quality changes, thus obtaining the image quality change trend of each local area within the key reading area of ​​the instrument.

[0191] The high-frequency image acquisition unit can be understood as a hardware module capable of continuously capturing images at a high frame rate, such as a high-speed camera or a vision system equipped with a high frame rate sensor. Its purpose is to capture subtle and rapid changes in image quality within key reading areas of the instrument, providing sufficient sample data density for subsequent quality trend analysis. Preset-frequency continuous image acquisition refers to continuously acquiring images at fixed, pre-set time intervals or frame rates to ensure the stability and continuity of the data stream.

[0192] Local image quality metrics can include, but are not limited to, local contrast, edge sharpness, brightness uniformity, and noise level. The real-time calculation of these metrics aims to quantify the image quality at different locations within the key reading area of ​​the instrument. For example, local contrast can be obtained by calculating the standard deviation or the difference between the maximum and minimum local pixel grayscale values; edge sharpness can be evaluated using edge detection operators such as Sobel and Canny combined with gradient magnitude; and brightness uniformity can be measured by calculating the mean and variance of brightness in a local area.

[0193] Time series analysis methods can employ various techniques, such as moving averages, exponential smoothing, and Kalman filtering. These methods aim to filter out transient, random noise interference from real-time calculated local image quality metrics, thereby revealing the true and underlying trends in image quality over time. Smoothing processes can avoid misjudgments caused by occasional interference (such as brief light fluctuations), making the assessment of image quality change trends more accurate and robust.

[0194] Optionally, the steps for continuously monitoring the image quality change trends of various local areas within the key reading area of ​​the instrument include:

[0195] When instantaneous optical distortion occurs in a high-frequency image, the specific spectral light source carried by the marine inspection robot is controlled to alternately irradiate the glass cover surface of the instrument to be inspected with a preset frequency and wavelength sequence.

[0196] Using a polarized light imaging unit, image frames of key reading areas of the instrument are acquired at different polarization angles;

[0197] The response differences of image features corresponding to each image frame under different spectral bands and the changes in polarization characteristics of interference regions in the image under different polarization angles were analyzed.

[0198] Based on response differences and changes in polarization characteristics, identify and distinguish dynamic optical effects from visual features caused by the instrument's inherent structure or background environment;

[0199] If the identification result is salt crystals or oil film, the image quality assessment weight of the corresponding area is adjusted according to the trend of optical property changes of salt crystals or oil film, and the glass cover cleaning module is triggered to process it.

[0200] Specifically, when the marine inspection robot detects transient optical distortions in the images acquired by its high-frequency image acquisition unit, such as localized brightness anomalies, blurring, or reflections, the system immediately activates an active intervention mechanism. A specific spectral light source mounted on the robot is activated and alternately illuminates the glass surface of the instrument under inspection according to a preset frequency and wavelength sequence, such as from visible light to near-infrared bands. This multispectral illumination aims to utilize the unique absorption, reflection, or transmission characteristics of different materials at different wavelengths to obtain richer image information.

[0201] Simultaneously, the polarization imaging unit is activated to acquire image frames of key reading areas of the instrument at multiple preset polarization angles (e.g., 0 degrees, 45 degrees, 90 degrees, and 135 degrees). Polarization imaging can reveal the microstructure, roughness, and presence of specific types of thin films (such as oil films and water films) on the surface of an object, because these substances have different effects on the polarization state of light.

[0202] The system then analyzes image frames acquired at different spectral bands to identify differences in the response of image features (such as brightness, contrast, and texture). For example, some interference may be more pronounced or disappear at specific wavelengths. Simultaneously, it analyzes the changes in the polarization characteristics of the interference region in the image at different polarization angles, such as changes in the degree of polarization and the polarization angle.

[0203] Based on these differences in spectral response and variations in polarization characteristics, the system can intelligently identify and distinguish between dynamic optical effects (such as transient glare, reflection, and water droplets) and visual features caused by the inherent structure of the instrument itself (such as scale lines and pointers) or the background environment (such as cabin lighting). This ability to differentiate is crucial for accurately determining the nature of interference.

[0204] Furthermore, if the identification results clearly indicate that the interference is caused by specific contaminants (such as salt crystals or oil films), the system will dynamically adjust the image quality assessment weights for the corresponding affected areas based on the unique optical characteristics of these contaminants (e.g., the difference in optical performance of salt crystals under dry and wet conditions, or the change in interference fringes of oil films under different lighting conditions). This means that in subsequent reading identification processes, the image data of these contaminated areas will be assigned different reliability weights. In addition, to eliminate or reduce this interference, the system will automatically trigger the glass cover cleaning module integrated on the marine inspection robot to clean the surface of the instrument glass cover to restore a clear visual environment.

[0205] This application also discloses a marine inspection robot control system, used to execute the control of a marine inspection robot, combined with... Figure 3 As shown, the marine inspection robot control system 1 includes:

[0206] The personalized feature acquisition module 11 is used to acquire the personalized feature information of the corresponding instrument before inspecting the ship's instruments, and to associate and store the personalized feature information with the corresponding instrument; the personalized feature information includes information on permanent physical defects of the instrument panel and / or pointers.

[0207] The aging status acquisition module 12 is used to acquire the actual aging status of the corresponding instrument and update the personalized feature information according to the actual aging status.

[0208] The instrument identification module 13 is used to identify the identity of the instrument to be inspected during the inspection process of the marine inspection robot.

[0209] The personalized feature retrieval module 14 is used to retrieve personalized feature information corresponding to the identity of the instrument to be inspected from the pre-stored personalized feature information database.

[0210] The instrument image acquisition module 15 is used to acquire images of the instruments currently to be inspected.

[0211] The interference elimination module 16 is used to process the image of the instrument to be inspected based on the retrieved personalized feature information, so as to eliminate or reduce the interference of permanent physical defects on the reading recognition.

[0212] The instrument reading extraction module 17 is used to extract the readings of the instrument to be inspected based on the processed image of the instrument to be inspected.

[0213] The reading result judgment module 18 is used to determine whether the reading of the instrument to be inspected is abnormal based on the reading of the instrument to be inspected and the retrieved personalized feature information, and to obtain the reading judgment result.

[0214] Specifically, the personalized feature acquisition module is configured to acquire personalized feature information of the corresponding instruments before inspecting them, and then associate and store this personalized feature information with the corresponding instruments. Personalized feature information may include information on permanent physical defects on the instrument panel and / or pointers. For example, this module can manually input information by having operators visually inspect each instrument during the initial robot deployment, recording scratches, stains, and faded areas on the panel, or rust and bending on the pointers, and then associating this information with the corresponding instrument number. Alternatively, the module can use a high-resolution camera to acquire initial images of the instruments and automatically identify and extract this permanent physical defect information using built-in image analysis software. For example, it can identify scratches and stains using edge detection and texture analysis algorithms, and identify faded areas using color analysis. This information is then stored in a database as structured data and associated with the corresponding instrument identity (such as instrument number, installation location, etc.).

[0215] The aging condition acquisition module is configured to acquire the actual aging condition of the corresponding instrument and update personalized feature information based on the actual aging condition. For example, this module can record the blurring degree of the instrument glass cover, the wear of the dial markings, and the degree of corrosion of the pointers through periodic manual inspections, and add these aging conditions as new personalized feature information to the database. Alternatively, the module can use sensors mounted on a robot, such as image analysis algorithms, to periodically evaluate the light transmittance of the instrument glass cover, the clarity of the dial markings, and the integrity of the pointers, and use these evaluation results as the actual aging condition. Based on these results, the module can automatically update or correct the stored personalized feature information. For example, if the blurring degree of the glass cover is detected to be increasing, a "glass cover blurring" mark is added to the personalized feature information, and its degree is recorded.

[0216] The instrument identification module is configured to identify the instrument to be inspected during the inspection process of a marine inspection robot. For example, this module can identify the instrument by placing QR codes or RFID tags near each instrument, which the robot scans. Alternatively, the module can utilize image recognition technology, employing a pre-trained deep learning model to identify the instrument's appearance features (such as shape, color, branding, etc.) to determine its identity.

[0217] The personalized feature retrieval module is configured to retrieve personalized feature information corresponding to the identity of the instrument to be inspected from a pre-stored personalized feature information database. For example, after the instrument identification module identifies the identity of the instrument to be inspected, the personalized feature retrieval module will query and extract the personalized feature information associated with the instrument from the database based on the identity information, including its permanent physical defects and the latest aging status.

[0218] The instrument image acquisition module is configured to acquire images of the instruments currently under inspection. For example, when an industrial camera mounted on a marine inspection robot inspects an instrument, it will take one or more images of the instrument under inspection. These images can be visible light images or other spectral images.

[0219] The interference cancellation module is configured to process the image of the instrument to be inspected based on retrieved personalized feature information to eliminate or reduce the interference of permanent physical defects on reading recognition. For example, if the personalized feature information indicates scratches on the instrument panel, the module can use image inpainting algorithms to fill pixels or synthesize textures in the scratched areas of the image to eliminate the interference of scratches on reading recognition. If the personalized feature information indicates stains on the instrument glass cover, the module can use image enhancement algorithms, such as contrast stretching and brightness adjustment, to reduce the impact of stains on image clarity.

[0220] The instrument reading extraction module is configured to extract the reading of the instrument to be inspected based on a processed image of the instrument. For example, this module can utilize traditional image processing algorithms, such as Hough transform to detect the pointer or circular detection to identify the instrument panel, and then calculate the reading based on the relative position of the pointer and the scale lines. Alternatively, the module can utilize a deep learning model, such as a convolutional neural network (CNN), to directly identify the instrument panel, scale lines, and pointer from the processed image and output the reading.

[0221] The reading result judgment module is configured to determine whether the reading of the instrument under inspection is abnormal based on the current reading and retrieved personalized feature information, and obtain the reading judgment result. For example, this module can compare the retrieved reading with a preset normal reading range; if it exceeds the range, it is judged as abnormal. Simultaneously, it incorporates personalized feature information; for example, if personalized feature information indicates that the instrument has severe aging, which may cause reading deviations, this deviation will be considered when judging abnormalities to avoid false alarms.

[0222] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A control method for a marine inspection robot, characterized in that, include: Before inspecting the ship's instruments, obtain the personalized feature information of the corresponding instruments and associate and store the personalized feature information with the corresponding instruments; the personalized feature information includes information on permanent physical defects of the instrument panel and / or pointers; Obtain the actual aging status of the corresponding instrument, and update the personalized feature information based on the actual aging status; During the inspection process of the marine inspection robot, the identity of the instrument to be inspected is identified. Retrieve personalized feature information corresponding to the identity of the instrument to be inspected from the pre-stored personalized feature information database; Acquire an image of the instrument currently under inspection; Based on the retrieved personalized feature information, the image of the instrument to be inspected is processed to eliminate or reduce the interference of the permanent physical defect information on the reading recognition; Based on the processed image of the instrument to be inspected, extract the reading of the instrument to be inspected. Based on the current reading of the instrument to be inspected and the retrieved personalized feature information, determine whether the current reading of the instrument to be inspected is an abnormal reading, and obtain the reading judgment result. The step of determining whether the reading of the instrument to be inspected is abnormal based on the current reading and the retrieved personalized feature information, and obtaining the reading determination result, includes: Obtain the readings of the instruments currently to be inspected; Obtain personalized feature information corresponding to the identity of the instrument currently to be inspected; Calculate the confidence level of the current instrument reading to be inspected, and perform threshold comparison on the confidence level of the reading to obtain the confidence level comparison result; When the confidence comparison result is lower than a preset threshold, the illumination mode of the image of the instrument to be inspected is switched. During the switching of illumination modes, several image frames of the instrument to be inspected are continuously acquired. Compare the dynamic changes of visual features of suspected interference areas in all image frames under different lighting modes; Based on the dynamic changes corresponding to visual features, the interference area is identified and the corresponding interference nature information is determined; Based on the interference nature information, determine whether the reading of the instrument to be inspected is an abnormal reading, and obtain the reading judgment result; The step of determining whether the reading of the instrument to be inspected is abnormal based on the interference nature information, and obtaining the reading determination result, includes: When the interference information is instantaneous optical distortion and the degree of overlap between the interference visual performance and the key reading area of ​​the instrument to be inspected reaches a preset value, the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm is calculated. Adjust the confidence level of the reading of the instrument to be inspected based on the contribution weight. If the adjusted reading confidence level is still lower than the preset threshold, the reading of the instrument to be inspected is re-extracted by weighted fusion of the features of the undisturbed area, and the reading re-extraction result is obtained. Based on the interference nature information and the reading re-extraction result, it is determined whether the reading of the instrument to be inspected is an abnormal reading, and the reading judgment result is obtained.

2. The control method for a marine inspection robot according to claim 1, characterized in that, When the interference information is instantaneous optical distortion and the degree of overlap between the interference visual appearance and the key reading area of ​​the instrument to be inspected reaches a preset value, the step of calculating the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm includes: Determine the overlap range between the interference area and the critical reading area of ​​the instrument to be inspected; Based on the overlap range, the local image quality of pixels within the overlap region is evaluated; the local image quality includes local contrast, edge sharpness, and brightness fluctuation amplitude. Based on the mapping relationship between the interference properties information and the sensitivity of the reading recognition algorithm, the weight of pixels in the overlapping area in the reading recognition algorithm is adjusted. The overlapping region is divided into sub-regions; The pixel contribution weights of each sub-region are calculated and weighted fusion is performed to obtain the contribution weights of the pixel values ​​of the interference region to the output of the reading recognition algorithm.

3. The control method for a marine inspection robot according to claim 1, characterized in that, If the adjusted reading confidence level is still lower than the preset threshold, the step of re-extracting the reading of the instrument to be inspected by weighted fusion of features from the undisturbed area to obtain the reading re-extraction result includes: The system continuously acquires images of the instruments to be inspected, resulting in a preset number of frames. Based on the dynamic change characteristics of the instantaneous optical distortion region corresponding to the interference nature information, the possible distribution range of the corresponding interference is determined; Real-time identification of undisturbed areas in the preset number of frames; Based on the possible distribution range and the undisturbed area in the preset frame number image, adjust the area of ​​interest for reading extraction; Local reading features are extracted from the undisturbed area in the preset frame number image, and the local reading features are weighted and fused to re-extract the reading of the instrument to be inspected, thus obtaining the reading re-extraction result; Evaluate the confidence level of the readings obtained from the re-extraction results.

4. The control method for a marine inspection robot according to claim 3, characterized in that, The step of evaluating the confidence level of the reading re-extraction results includes: Identify the overlapping areas of permanent physical defects and transient optical distortions within the key reading areas of the instrument; The degree of image feature degradation at different locations within the overlapping region is analyzed; the degree of degradation includes reduced local contrast, blurred edges, and brightness fluctuations. Based on the degree of degradation, the interference weight of each pixel in the overlapping area for reading recognition is determined; By combining the interference weights and the personalized feature information, compensatory processing is performed on the reading features within the overlapping region; Based on the reading characteristics after compensatory processing, the local confidence level of the re-extracted readings is calculated; The local confidence scores are weighted and fused to obtain the overall confidence score of the reading re-extraction result, which is recorded as the reading confidence score of the reading re-extraction result.

5. The control method for a marine inspection robot according to claim 4, characterized in that, The step of weightedly fusing the local confidence scores to obtain the overall confidence score of the reading re-extraction result and recording it as the reading confidence score of the reading re-extraction result includes: Continuously monitor the image quality change trend of each local area within the key reading area of ​​the instrument; Based on the rate and magnitude of change in the image quality change trend, and combined with the sensitivity of the image quality change to the reading recognition algorithm, the weight of the local confidence score is dynamically adjusted. Based on the frequency and duration of the current illumination mode switching, as well as the possible distribution range of instantaneous optical distortion, the weight adjustment of the local confidence is periodically calibrated. Based on the adjusted weights of the local confidence scores, the local confidence scores are weighted and fused to obtain the overall confidence score of the reading re-extraction result, which is recorded as the reading confidence score of the reading re-extraction result.

6. The control method for a marine inspection robot according to claim 5, characterized in that, The steps for continuously monitoring the image quality change trend of each local area within the key reading area of ​​the instrument include: The high-frequency image acquisition unit integrated on the marine inspection robot is activated to continuously acquire images of key instrument reading areas at a preset frequency, thus obtaining high-frequency images. Real-time calculation of local image quality indicators in the high-frequency image; By using time series analysis, the local image quality indicators are processed to smooth out instantaneous noise and highlight the true trend of image quality changes, thereby obtaining the image quality change trend of each local area within the key reading area of ​​the instrument.

7. The control method for a marine inspection robot according to claim 6, characterized in that, The steps for continuously monitoring the image quality change trend of each local area within the key reading area of ​​the instrument include: When a transient optical distortion occurs in the high-frequency image, the specific spectral light source carried by the marine inspection robot is controlled to alternately irradiate the surface of the glass cover of the instrument to be inspected with a preset frequency and wavelength sequence. Using a polarized light imaging unit, image frames of key reading areas of the instrument are acquired at different polarization angles; The response differences of image features corresponding to each image frame under different spectral bands and the changes in polarization characteristics of interference regions in the image under different polarization angles were analyzed. Based on the response differences and the changes in polarization characteristics, identify and distinguish dynamic optical effects from visual features caused by the instrument's inherent structure or background environment; If the identification result is salt crystals or oil film, the image quality assessment weight of the corresponding area is adjusted according to the trend of optical property changes of salt crystals or oil film, and the glass cover cleaning module is triggered to process it.

8. A control system for a marine inspection robot, used to execute control of a marine inspection robot, characterized in that, include: The personalized feature acquisition module is used to acquire personalized feature information of the corresponding instruments before inspecting the ship's instruments, and to associate and store the personalized feature information with the corresponding instruments; the personalized feature information includes information on permanent physical defects of the instrument panel and / or pointers; The aging status acquisition module is used to acquire the actual aging status of the corresponding instrument and update the personalized feature information based on the actual aging status. The instrument identification module is used to identify the identity of the instrument to be inspected during the inspection process of the marine inspection robot. The personalized feature retrieval module is used to retrieve personalized feature information corresponding to the identity of the instrument to be inspected from a pre-stored personalized feature information database. The instrument image acquisition module is used to acquire images of the instruments currently to be inspected; The interference elimination module is used to process the image of the instrument to be inspected based on the retrieved personalized feature information, so as to eliminate or reduce the interference of the permanent physical defect information on the reading recognition. The instrument reading extraction module is used to extract the readings of the instrument to be inspected based on the processed image of the instrument to be inspected. The reading result judgment module is used to determine whether the reading of the instrument to be inspected is abnormal based on the reading of the instrument to be inspected and the retrieved personalized feature information, and to obtain the reading judgment result. It is also used to obtain the readings of the instruments currently under inspection; Obtain personalized feature information corresponding to the identity of the instrument currently to be inspected; Calculate the confidence level of the current instrument reading to be inspected, and perform threshold comparison on the confidence level of the reading to obtain the confidence level comparison result; When the confidence comparison result is lower than a preset threshold, the illumination mode of the image of the instrument to be inspected is switched. During the switching of illumination modes, several image frames of the instrument to be inspected are continuously acquired. Compare the dynamic changes of visual features of suspected interference areas in all image frames under different lighting modes; Based on the dynamic changes corresponding to visual features, the interference area is identified and the corresponding interference nature information is determined; Based on the interference nature information, determine whether the reading of the instrument to be inspected is an abnormal reading, and obtain the reading judgment result; It is also used to calculate the contribution weight of the pixel value of the interference area to the output of the reading recognition algorithm when the interference nature information is instantaneous optical distortion and the degree of overlap between the interference visual performance and the key reading area of ​​the instrument to be inspected reaches a preset value. Adjust the confidence level of the reading of the instrument to be inspected based on the contribution weight. If the adjusted reading confidence level is still lower than the preset threshold, the reading of the instrument to be inspected is re-extracted by weighted fusion of the features of the undisturbed area, and the reading re-extraction result is obtained. Based on the interference nature information and the reading re-extraction result, it is determined whether the reading of the instrument to be inspected is an abnormal reading, and the reading judgment result is obtained.

Images