Visual inspection system for ship cleaning robot

Abstract

The invention discloses a visual inspection system for a ship cleaning robot, and relates to the technical field of visual inspection, and the system comprises an image correction unit which is used for obtaining an image quality index of a ship and a visual influence index of the surrounding environment of the ship and correcting a visual image of the ship when a bad ship image or a severe environment condition occurs; according to the invention, by calculating the image quality index and automatically dividing the qualified ship image and the bad ship image, the image quality problem can be identified in time, and the judgment error caused by the defect of the image is avoided; besides, the environmental analysis unit quantifies visual influence indexes and performs suitability and hostile division on the environment, fully considers the interference of illumination, salt mist and wind speed environmental factors in actual operation, and enhances the adaptability of the system in a complex marine environment. And when the image quality is poor or the environment is severe, the image correction unit dynamically calculates a correction coefficient based on the image quality index and the visual influence index, and performs self-adaptive correction on the visual image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual detection, and particularly relates to a visual detection system for a ship cleaning robot. BACKGROUND

[0002] During long-term sailing, foreign matters such as marine organisms (e.g., barnacles, oysters), oil stains and rust are easily attached to the surface of a ship, which can increase the sailing resistance of the ship, reduce the propulsion efficiency, accelerate the corrosion of the ship body and shorten the service life of the ship. As an efficient ship body maintenance device, the detection accuracy of the visual detection system of a ship cleaning robot directly determines the pertinence and effectiveness of the cleaning operation.

[0003] However, the visual detection technology of the conventional ship cleaning robot mainly relies on a single image feature to judge foreign matters, and the influence of the ship operation environment (e.g., light intensity, sea wind humidity and sea wave shaking) on the visual image quality is not fully considered, which can easily cause image distortion, foreign matter misjudgment or omission in a complex marine environment.

[0004] In view of the above technical defects, the present application provides a solution. SUMMARY

[0005] The present application aims at solving the problem that the visual detection technology of the conventional ship cleaning robot mainly relies on a single image feature to judge foreign matters, and the influence of the ship operation environment on the visual image quality is not fully considered, which can easily cause image distortion, foreign matter misjudgment or omission in a complex marine environment.

[0006] To achieve the above object, the present application adopts the following technical scheme: a visual detection system for a ship cleaning robot, comprising a data acquisition unit, a foreign matter judgment unit, a quality analysis unit, an environment analysis unit and an image correction unit.

[0007] The data acquisition unit is used for acquiring image data of the ship and environment data around the ship through a sensor array, and pre-processing the acquired data, and sending the pre-processed image data to the foreign matter judgment unit and the quality analysis unit, and sending the environment data to the environment analysis unit.

[0008] The foreign matter judgment unit is used for receiving the image data, and constructing a ship attached foreign matter judgment model through a convolutional neural network to identify the attached foreign matter condition on the ship.

[0009] The quality analysis unit is used for receiving the image data, performing analysis and calculation to obtain an image quality index of the ship, and dividing the image of the ship into qualified ship images and poor ship images.

[0010] The environmental analysis unit is configured to receive environmental data, perform analysis calculation, obtain a visual impact index of the environment around the ship, and divide the environment around the ship into a severe environment and a suitable environment.

[0011] The image correction unit is configured to obtain an image quality index of the ship and the visual impact index of the environment around the ship when an undesirable ship image or a severe environment occurs, and perform analysis calculation to correct the visual image of the ship.

[0012] Further, the image data includes a visual image of the ship, pixel values of the ship image, gray values, and signal-to-noise ratio data of the ship image, and the environmental data includes light intensity, salt mist concentration, and wind speed data of the environment around the ship.

[0013] Further, the process of constructing the ship attached foreign matter judgment model is as follows:

[0014] S11, collect a plurality of historical morphological images of different ship attached foreign matters as a data set, and randomly divide the data set into a training set and a test set, and label the corresponding foreign matter species on each morphological image in the training set as a label;

[0015] S12, constructing a ship attached foreign matter judgment model based on a convolutional neural network, training the ship attached foreign matter judgment model using the training set, and testing the ship attached foreign matter judgment model using the test set to obtain a test qualified ship attached foreign matter judgment model;

[0016] S13, capturing a real-time ship image of a surface of a ship to be cleaned through a visual sensor on a cleaning robot, and inputting the real-time ship image into the test qualified ship attached foreign matter judgment model to judge the situation of the attached foreign matter on the ship to be identified.

[0017] Further, the calculation process of the image quality index of the ship is as follows:

[0018] S21, obtaining pixel values, gray values, and signal-to-noise ratio data of the ship image and performing analysis calculation;

[0019] S22, calculating the image quality index Q of the ship according to the following formula:

[0020]

[0021] Wherein, M is the horizontal size of the ship image, N is the vertical size of the ship image, G (x,y) is the original pixel value of the ship image at coordinate (x, y), G (x+1,y) is the original pixel value of the ship image at coordinate (x+1, y), I is the gray value of the ship image at coordinate (x, y), and These are the gradients of the ship image in the x and y directions, respectively, and SNR is the signal-to-noise ratio of the ship image.

[0022] S23. Obtain the preset image quality threshold Q. th A comparative analysis was conducted with the image quality index Q of the ship. When Q > Q th If the image quality is high, it indicates that the image is of good quality and is classified as a qualified image, requiring no correction of the visual image.

[0023] S24, when Q≤Q th If the image quality is poor, it indicates that the ship's image is of low quality and is classified as a defective ship image, requiring correction of the ship's visual image.

[0024] Furthermore, the calculation process for the visual impact index of the ship's surrounding environment is as follows:

[0025] S31. Acquire and analyze data on the light intensity, salt spray concentration, and wind speed of the environment surrounding the ship.

[0026] S32. Calculate the visual impact index R of the environment surrounding the ship according to the following formula:

[0027]

[0028] Where L is the light intensity of the environment surrounding the ship, L ref L is the preset standard light intensity. max L is the preset upper limit of light intensity. min C represents the preset lower limit of light intensity, and C is the salt spray concentration in the environment surrounding the ship. ref The preset standard salt spray concentration is given by [reference to standard salt spray concentration], and `wind` represents the wind speed in the surrounding environment of the ship. ref The preset standard ambient wind speed, wind avg The historical average wind speed of the environment surrounding the ship is given by , a is the preset weighting coefficient for light intensity, b is the preset weighting coefficient for salt spray concentration, and c is the preset weighting coefficient for wind speed, and a+b+c=1.

[0029] S33. Obtain the preset visual impact threshold R th A comparative analysis was conducted with the visual impact index R of the ship's surrounding environment. When R ≤ R th If the environment around the ship is good, it indicates that the environment around the ship is suitable and no correction is needed to the visual image of the ship.

[0030] S34, When R>R th If the environment around the ship is deemed harsh, it indicates that the ship's surroundings are in a difficult or unfavorable condition, and the visual image of the ship needs to be corrected.

[0031] Further, the process of correcting the visual image of the ship is as follows:

[0032] S41, acquire the image quality index of the ship and the visual influence index data of the environment around the ship and analyze and calculate;

[0033] S42, calculate the correction coefficient k of the visual image of the ship according to the following formula:

[0034]

[0035] Wherein, Q is the image quality index of the ship, R is the visual influence index of the environment around the ship, Q th is the image quality threshold of the ship, R th is the visual influence threshold of the environment around the ship, and a is the preset adjustment coefficient of the image, and b is the preset adjustment coefficient of the visual influence;

[0036] S43, calculate the pixel value of the corrected visual image of the ship according to the following formula: G (x,y) , wherein k is the correction coefficient of the visual image of the ship, G (x,y) is the original pixel value of the ship image at coordinates (x, y);

[0037] S44, send the corrected visual image of the ship to the ship attached foreign matter judgment model to judge the attached foreign matter on the ship.

[0038] As described above, due to the adoption of the above technical scheme, the beneficial effects of the present application are:

[0039] The visual detection system for the ship cleaning robot comprehensively acquires ship image data and surrounding environment data through the data acquisition unit, realizes multi-dimensional information acquisition, provides a comprehensive data basis for subsequent analysis, and thus significantly improves the perception ability of the system; then the foreign matter judgment unit constructs a ship attached foreign matter judgment model by using a convolutional neural network, effectively identifies various foreign matter types, greatly improves the accuracy and robustness of foreign matter detection, and the quality analysis unit can identify image quality problems in time and avoid judgment errors caused by defects in the image itself; in addition, the environment analysis unit quantifies the visual influence index and divides the environment into suitable and adverse, fully considers the interference of light, salt mist and wind speed environmental factors in actual operation, and enhances the adaptability of the system in complex marine environment; and the image correction unit dynamically calculates the correction coefficient based on the image quality index and the visual influence index when the image quality is poor or the environment is bad, and adaptively corrects the visual image, thereby effectively restoring the image details and improving the image clarity, ensuring the reliability of subsequent foreign matter judgment. Attached Figure Description

[0040] Figure 1 A schematic diagram of the system flow of the present invention is shown. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example:

[0043] like Figure 1 As shown, a vision inspection system for a ship cleaning robot first uses a data acquisition unit to collect image data of the ship and environmental data of the surrounding environment through a sensor array (including a vision sensor, a light sensor, an electrochemical sensor, and a wind speed sensor). The collected data is preprocessed, and the preprocessed image data is sent to a foreign object detection unit and a quality analysis unit, while the environmental data is sent to an environmental analysis unit. It should be noted that the image data includes the ship's visual image, pixel values, grayscale values, and signal-to-noise ratio data; the environmental data includes the light intensity, salt spray concentration, and wind speed data of the surrounding environment. Simultaneously, the foreign object detection unit receives the image data and constructs a foreign object detection model for the ship using a convolutional neural network to identify the foreign objects attached to the ship.

[0044] The process of constructing a foreign object detection model for ships is as follows:

[0045] S11. Collect historical morphological images of foreign objects attached to multiple different ships as a dataset, and randomly divide the dataset into a training set and a test set. Label the corresponding foreign object types on each morphological image in the training set as labels.

[0046] S12. Construct a foreign object detection model for ships based on a convolutional neural network. Train the foreign object detection model using a training set and test the foreign object detection model using a test set to obtain a qualified foreign object detection model for ships.

[0047] S13. Take real-time images of the ship surface to be cleaned using the vision sensor on the cleaning robot, and input the real-time images into the qualified ship foreign object judgment model to determine the situation of foreign objects attached to the ship.

[0048] Then, the image data is received by the quality analysis unit, analysis calculation is performed, the image quality index of the ship is obtained, and the image of the ship is divided into qualified ship image and poor ship image.

[0049] The calculation process of the image quality index of the ship is as follows:

[0050] S21, pixel values, gray values of the ship image and signal-to-noise ratio data of the ship image are obtained and analyzed and calculated;

[0051] S22, the image quality index Q of the ship is calculated according to the following formula:

[0052]

[0053] Wherein, M is the horizontal size of the ship image, N is the vertical size of the ship image, G (x,y) is the original pixel value of the ship image at coordinate (x, y), G (x+1,y) is the original pixel value of the ship image at coordinate (x+1, y), I is the gray value of the ship image at coordinate (x, y), and are the gradients of the ship image in x direction and y direction respectively (calculated by Sobel operator and other edge detection operators), and SNR is the signal-to-noise ratio of the ship image;

[0054] S23, the preset image quality threshold Q th is compared and analyzed with the image quality index Q of the ship, when Q>Q th , it indicates that the image quality of the ship is high, and the qualified ship image is divided, and the visual image of the ship does not need to be corrected;

[0055] S24, when Q≤Q th , it indicates that the image quality of the ship is low, and the poor ship image is divided, and the visual image of the ship needs to be corrected.

[0056] Then, the environmental analysis unit receives the environmental data, performs analysis calculation, obtains the visual influence index of the environment around the ship, and divides the environment around the ship into severe environment and suitable environment;

[0057] The calculation process of the visual influence index of the environment around the ship is as follows:

[0058] S31, the light intensity, salt mist concentration and wind speed data of the environment around the ship are obtained and analyzed and calculated;

[0059] S32, the visual influence index R of the environment around the ship is calculated according to the following formula:

[0060]

[0061] wherein, L is the light intensity of the surrounding environment of the ship, L ref is the preset standard light intensity, L max is the preset upper limit value of the light intensity, L min is the preset lower limit value of the light intensity, C is the salt mist concentration of the surrounding environment of the ship, C ref is the preset standard salt mist concentration, wind is the wind speed of the surrounding environment of the ship, wind ref is the preset standard environmental wind speed, wind avg is the historical average wind speed of the surrounding environment of the ship, a is the weight coefficient of the preset light intensity, b is the weight coefficient of the preset salt mist concentration, c is the weight coefficient of the preset wind speed, and a+b+c=1.

[0062] S33, acquiring a preset visual influence threshold R th , and comparing and analyzing the visual influence index R of the surrounding environment of the ship, when R≤R th , it means that the surrounding environment of the ship is good, and the surrounding environment of the ship is divided into suitable environment, and the visual image of the ship does not need to be corrected.

[0063] S34, when R>R th , it means that the surrounding environment of the ship is bad, and the surrounding environment of the ship is divided into bad environment, and the visual image of the ship needs to be corrected.

[0064] Finally, through the image correction unit, when the bad ship image or the bad environment occurs, the image quality index of the ship and the visual influence index of the surrounding environment of the ship are acquired and analyzed and calculated, and the visual image of the ship is corrected.

[0065] The process of correcting the visual image of the ship is as follows:

[0066] S41, acquiring the image quality index of the ship and the visual influence index data of the surrounding environment of the ship and analyzing and calculating;

[0067] S42, calculating the correction coefficient k of the visual image of the ship according to the following formula:

[0068]

[0069] wherein, Q is the image quality index of the ship, R is the visual influence index of the surrounding environment of the ship, Q th is the image quality threshold of the ship, R thThe visual influence threshold of the surrounding environment of the ship, alpha is a preset image adjustment coefficient, beta is a preset visual influence adjustment coefficient, wherein the preset image adjustment coefficient alpha and the preset visual influence adjustment coefficient beta are determined through a large amount of experimental data;

[0070] S43, calculate the pixel value of the corrected visual image of the ship according to the following formula Wherein, k is the correction coefficient of the visual image of the ship, G (x,y) Is the original pixel value of the ship image at coordinates (x, y);

[0071] S44, send the corrected visual image of the ship to the ship attached foreign matter judgment model, judge the attached foreign matter on the ship, if the ship surface is attached with foreign matter, then dispatch the ship cleaning robot to clean, otherwise, no need to clean.

[0072] The present application comprehensively obtains ship image data and surrounding environment data through the data acquisition unit, realizes multi-dimensional information acquisition, provides a comprehensive data basis for subsequent analysis, thereby significantly improving the perception ability of the system; further, the foreign matter judgment unit utilizes the convolutional neural network to construct the ship attached foreign matter judgment model, effectively identifies various foreign matter types, greatly improves the accuracy and robustness of foreign matter detection, and the quality analysis unit calculates the image quality index and automatically divides qualified and unqualified ship images, which can timely identify image quality problems and avoid judgment errors caused by image defects; in addition, the environment analysis unit quantifies the visual influence index and divides the environment into suitable and adverse, fully considers the interference of environmental factors such as light, salt fog and wind speed in actual operation, and enhances the adaptability of the system in complex marine environment; and the image correction unit dynamically calculates the correction coefficient based on the image quality index and the visual influence index when the image quality is poor or the environment is adverse, and adaptively corrects the visual image, thereby effectively restoring image details and improving image clarity, and ensuring the reliability of subsequent foreign matter judgment.

[0073] The setting of the size of the interval and the threshold is for easy comparison, and the size of the threshold depends on the amount of sample data and the base number set by the person skilled in the art for each group of sample data; as long as it does not affect the proportional relationship of the parameters and the quantized values.

[0074] The above formulas are dimensionless values, and the formulas are obtained by software simulation of a large amount of data to obtain a formula closest to the actual situation, and the preset parameters in the formula are set by the person skilled in the art according to the actual situation;

[0075] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art, according to the technical solution and inventive concept of the present application, makes equivalent replacement or change within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A vision inspection system for a ship cleaning robot, characterized in that, include: The system includes a data acquisition unit, a foreign object detection unit, a quality analysis unit, an environmental analysis unit, and an image correction unit. The data acquisition unit is used to collect image data of the ship and environmental data around the ship through a sensor array, preprocess the collected data, and send the preprocessed image data to the foreign object detection unit and the quality analysis unit, and send the environmental data to the environmental analysis unit. The foreign object detection unit is used to receive image data and construct a foreign object detection model for ships through a convolutional neural network to identify foreign objects attached to the ship. The quality analysis unit is used to receive image data, perform analysis and calculation, obtain the image quality index of the ship, and classify the ship images into qualified ship images and poor ship images. The environmental analysis unit is used to receive environmental data, perform analysis and calculations, derive the visual impact index of the environment around the ship, and classify the environment around the ship into harsh environments and suitable environments. The image correction unit is used to obtain the image quality index of the ship and the visual impact index of the surrounding environment when poor ship images or adverse environmental conditions occur, and to perform analysis and calculation to correct the visual image of the ship.

2. The vision inspection system for a ship cleaning robot according to claim 1, characterized in that, The image data includes visual images of the ship, pixel values, grayscale values, and signal-to-noise ratio data of the ship images. The environmental data includes light intensity, salt spray concentration, and wind speed data of the environment surrounding the ship.

3. The vision inspection system for a ship cleaning robot according to claim 1, characterized in that, The process of constructing a foreign object detection model for ships is as follows: S11. Collect historical morphological images of foreign objects attached to multiple different ships as a dataset, and randomly divide the dataset into a training set and a test set. Label the corresponding foreign object types on each morphological image in the training set as labels. S12. Construct a foreign object detection model for ships based on a convolutional neural network. Train the foreign object detection model using a training set and test the foreign object detection model using a test set to obtain a qualified foreign object detection model for ships. S13. Take real-time images of the ship surface to be cleaned using the vision sensor on the cleaning robot, and input the real-time images into the qualified ship foreign object judgment model to determine the situation of foreign objects attached to the ship.

4. The vision inspection system for a ship cleaning robot according to claim 1, characterized in that, The calculation process for the image quality index of a ship is as follows: S21. Obtain the pixel values, grayscale values, and signal-to-noise ratio data of the ship image and perform analysis and calculation. S22. Calculate the image quality index Q of the ship according to the following formula: Where M is the horizontal dimension of the ship image, N is the vertical dimension of the ship image, and G... (x,y) G represents the original pixel value of the ship image at coordinates (x, y). (x+1,y) Let be the original pixel value of the ship image at coordinates (x+1, y), and let I be the grayscale value of the ship image at coordinates (x, y). and These are the gradients of the ship image in the x and y directions, respectively, and SNR is the signal-to-noise ratio of the ship image. S23. Obtain the preset image quality threshold Q. th A comparative analysis was conducted with the image quality index Q of the ship. When Q > Q th If the image quality is high, it indicates that the image is of good quality and is classified as a qualified image, requiring no correction of the visual image. S24, when Q≤Q th If the image quality is poor, it indicates that the ship's image is of low quality and is classified as a defective ship image, requiring correction of the ship's visual image.

5. The vision inspection system for a ship cleaning robot according to claim 1, characterized in that, The calculation process for the visual impact index of the environment surrounding a ship is as follows: S31. Acquire and analyze data on the light intensity, salt spray concentration, and wind speed of the environment surrounding the ship. S32. Calculate the visual impact index R of the environment surrounding the ship according to the following formula: Where L is the light intensity of the environment surrounding the ship, L ref L is the preset standard light intensity. max L is the preset upper limit of light intensity. min C represents the preset lower limit of light intensity, and C is the salt spray concentration in the environment surrounding the ship. ref The preset standard salt spray concentration is given by [reference to standard salt spray concentration], and `wind` represents the wind speed in the surrounding environment of the ship. ref The preset standard ambient wind speed, wind avg The historical average wind speed of the environment surrounding the ship is given by , a is the preset weighting coefficient for light intensity, b is the preset weighting coefficient for salt spray concentration, and c is the preset weighting coefficient for wind speed, and a+b+c=1. S33. Obtain the preset visual impact threshold R th A comparative analysis was conducted with the visual impact index R of the ship's surrounding environment. When R ≤ R th If the environment around the ship is good, it indicates that the environment around the ship is suitable and no correction is needed to the visual image of the ship. S34, When R>R th If the environment around the ship is deemed harsh, it indicates that the ship's surroundings are in a difficult or unfavorable condition, and the visual image of the ship needs to be corrected.

6. The vision inspection system for a ship cleaning robot according to claim 1, characterized in that, The process of correcting the visual image of a ship is as follows: S41. Acquire the image quality index of the ship and the visual impact index of the surrounding environment of the ship and perform analysis and calculation. S42. Calculate the correction factor k for the ship's visual image according to the following formula: Where Q is the image quality index of the ship, and R is the visual impact index of the environment surrounding the ship. th R is the image quality threshold for ships. th α is the visual impact threshold of the ship's surrounding environment, β is the preset adjustment coefficient of the image, and β is the preset adjustment coefficient of the visual impact. S43. Calculate the pixel value of the corrected visual image of the ship according to the following formula. Where k is the correction coefficient for the ship's visual image, and G (x,y) This represents the raw pixel value of the ship image at coordinates (x, y). S44. Send the corrected visual image of the ship to the ship foreign object judgment model to judge the foreign object situation on the ship.

Images