Method, device, terminal and storage medium for determining damage degree of ship components

Abstract

The present invention discloses a method, device, terminal, and storage medium for determining the degree of damage of a ship component. The method includes: obtaining images of the component to be assessed of the ship to be assessed and the control component of a reference ship from N perspectives, where N is a positive integer; based on a pre-trained classification model, inputting the images of the component to be assessed and the control component from N perspectives into the classification model for classification processing to obtain the probability of each image of the component to be assessed from the N perspectives being at each of M damage levels, where M is a positive integer; determining the damage level of the component to be assessed based on the probability of each image of the component to be assessed from the N perspectives being at each of the M damage levels, so as to determine the degree of damage of the component to be assessed. This solution, by using a deep learning method to perform ship observation value assessment (such as determining or assessing the degree of damage of a ship component), is conducive to improving the accuracy of the assessment results.

Description

Technical Field

[0001] The present invention belongs to the technical field of ship damage assessment, and specifically relates to a method, device, terminal and storage medium for determining the damage degree of ship components, and more particularly to a method, device, terminal and storage medium for assessing the damage degree of ship components based on multi-view salient feature recognition. Background Art

[0002] Ship valuation (such as determining or assessing the extent of depreciation of ship components) is widely used in both the shipping market and shipping finance and insurance. Objectively assessing ship value (such as determining or assessing the extent of depreciation of ship components) is crucial for facilitating smooth cooperation between ship buyers and sellers, as well as determining ship insurance limits and accident loss settlement amounts. Due to the unique circumstances of each ship, there is no unified methodology for ship valuation. Widely used mainstream methods include the cost approach, market approach, present value approach, and income approach. The cost approach determines the valuation of the vessel being assessed based on its current replacement cost, deducting various depreciations. The market approach identifies technically similar vessels in the recent ship market as reference vessels, analyzes and compares the differences between the vessel being assessed and the reference vessels, determines the impact of these differences on the vessel's value, and then adjusts the estimated value based on these differences. The income approach determines the value of the vessel being assessed by estimating the expected future earnings and converting them into present value. The present value method, based on the concept of present value in engineering economics, converts the annual capital recovery costs of a vessel from the valuation base date to its scrapping age into its present value in the year of the valuation base date, representing the vessel's asset valuation. Regardless of the valuation method used, the appearance and condition of the vessel and its components are a key factor influencing the vessel's final valuation. There are two main methods for assessing the visual value of a vessel and its components: one in which a ship appraiser conducts an on-site inspection of the object to be appraised, and the other in which a ship appraiser conducts an assessment based on images of the object to be appraised.

[0003] In the relevant schemes' methods for assessing the observed value of ship components (such as determining or assessing the extent of damage to ship components), the ship to be appraised and the ship appraiser are generally located in different geographical locations. The manpower and time costs of having the ship appraiser personally conduct on-site inspections of each ship to be appraised are high. This is especially true for insurance claims arising from the loss or sinking of a ship, as the on-site inspection and assessment method is less practical.

[0004] For the reasons mentioned above, it is becoming increasingly common to use videos and photos of ship components instead of on-site inspections. However, it is unavoidable that the subjective will of ship appraisers during manual evaluation will affect the evaluation results (such as the determination of the degree of damage to ship components or the results of the evaluation), resulting in non-objective evaluations and inaccurate value assessment results (such as the determination of the degree of damage to ship components or the results of the evaluation). Although there are regulations on the evaluation methods of ship appraisers, it is impossible to guarantee that different ship appraisers will achieve uniformity.

[0005] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0006] The object of the present invention is to provide a method, device, terminal and storage medium for determining the degree of damage of ship components, so as to solve the problem that in the method for evaluating the observation value of ship components (such as determining or evaluating the degree of damage of ship components) in related schemes, ship appraisers use videos and photos of ship components instead of on-site inspections, but cannot avoid the influence of the subjective will of the ship appraiser during manual evaluation on the evaluation results (such as the results of determining or evaluating the degree of damage of ship components), and cannot improve the accuracy of the evaluation results (such as the results of determining or evaluating the degree of damage of ship components). By using a deep learning method to perform ship observation value evaluation (such as determining or evaluating the degree of damage of ship components), the influence of the subjective will of the ship appraiser during manual evaluation on the evaluation results (such as the results of determining or evaluating the degree of damage of ship components) can be avoided, which is conducive to improving the accuracy of the evaluation results (such as the results of determining or evaluating the degree of damage of ship components).

[0007] The present invention provides a method for determining the degree of damage of a ship component, comprising: obtaining images of a component to be assessed of a ship to be assessed and a control component of a reference ship from N perspectives, where N is a positive integer; wherein, among the images of the component to be assessed from the N perspectives and the images of the control component from the N perspectives, the images of the component to be assessed and the images of the control component that are used for comparison are a pair of images obtained from the same perspective; based on a pre-trained classification model, the images of the component to be assessed and the control component from the N perspectives are input into the classification model to obtain the images of the component to be assessed and the control component from the N perspectives. In the images from the N perspectives, the probability that the damage degree of the component to be evaluated relative to the control component is at each of the M preset damage levels is compared with each image of the control component from the same perspective. This probability is recorded as the probability of each image in the images from the N perspectives of the component to be evaluated being at each of the M damage levels, where M is a positive integer. Based on the probability of each image in the images from the N perspectives of the component to be evaluated being at each of the M damage levels, the damage level of the component to be evaluated is determined to achieve determination of the damage degree of the component to be evaluated.

[0008] In some embodiments, the pre-trained classification model is obtained by training based on a deep neural network; wherein, the operation of training the classification model based on the deep neural network includes: generating training samples by manual annotation and / or simulation, wherein the training samples include images of the component to be assessed and the reference component of the ship from more than one perspective, and a given probability that the degree of damage of the component to be assessed relative to the reference component is within the M damage levels based on the comparison between each image of the component to be assessed and the image of the reference component from the same perspective; recording the given probability that the degree of damage of the component to be assessed relative to the reference component is within the M damage levels based on the comparison between each image of the component to be assessed and the image of the reference component from the same perspective as the given probability of each damage level in the M damage levels for each image in the images of the component to be assessed from the above perspectives; and recording the images of the component to be assessed and the reference component from more than one perspective in the training samples as the given probability of each damage level in the M damage levels. The deep neural network is trained as the input of the deep neural network to obtain the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training sample in the M damage levels; the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training sample in the M damage levels is recorded as the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the M damage levels; based on the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the M damage levels, combined with the loss function, the given probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training sample in the M damage levels is used as the output of the deep neural network, and the deep neural network is de-trained to obtain the deep neural network after training and de-training as the classification model.

[0009] In some embodiments, the deep neural network includes: a twin feature extraction network, a saliency network, and a fully connected network; wherein, the images of the component to be assessed and the reference component of the ship in the training sample from more than one perspective are used as inputs to the deep neural network, and the deep neural network is trained to obtain the training probability of each of the images of the component to be assessed in the training sample from more than one perspective for each of the M damage levels, including: extracting image features based on the input images of the component to be assessed and the reference component of the ship in the training sample from more than one perspective through the twin feature extraction network to obtain two sets of image features of the images of the component to be assessed and the reference component of the ship in the training sample from more than one perspective; and obtaining two sets of image features of the images of the component to be assessed and the reference component of the ship in the training sample from more than one perspective through the twin feature extraction network; and obtaining two sets of image features of the images of the component to be assessed and the reference component of the ship in the training sample from more than one perspective through the twin feature extraction network. The saliency network constructs an association of image features from more than one perspective based on two sets of image features of the image of the component to be assessed and the control component from more than one perspective in the training sample, thereby obtaining a set of aggregated features of the image of the component to be assessed and the control component from more than one perspective in the training sample. The fully connected network performs a fully connected process based on the set of aggregated features of the image of the component to be assessed and the control component from more than one perspective in the training sample, and outputs the probability of each image in the images of the component to be assessed from more than one perspective in the training sample for each of the M damage levels, as the training probability of each image in the images of the component to be assessed from more than one perspective in the training sample for each of the M damage levels.

[0010] In some embodiments, the damage grade of the component to be assessed is determined based on the probability of each damage grade among the M damage grades for each image in the N perspectives of the images of the component to be assessed, so as to determine the degree of damage of the component to be assessed, including: based on the probability of each damage grade among the M damage grades for each image in the N perspectives of the images of the component to be assessed, selecting a damage grade with the largest probability as the damage grade of the images of the N perspectives of the component to be assessed, or selecting damage grades corresponding to a group of probabilities with probabilities greater than a set value and taking a weighted average as the damage grade of the images of the N perspectives of the component to be assessed; based on the damage grade of each image of the component to be assessed in the damage grades of the images of the N perspectives of the component to be assessed, taking a weighted average of the damage grades of the N images of the component to be assessed to obtain the damage grade of the component to be assessed; and further, determining the value of the component to be assessed based on the damage grade of the component to be assessed and the value of the control component, which is recorded as the observed value of the component to be assessed.

[0011] Matching the above method, the present invention provides, on the other hand, a device for determining the degree of damage of a ship component, comprising: an image acquisition unit, configured to acquire images of the component to be assessed of the ship to be assessed and the control component of the reference ship from N perspectives, where N is a positive integer; wherein, among the images of the component to be assessed from N perspectives and the images of the control component from N perspectives, the images of the component to be assessed and the images of the control component that are compared with each other are a pair of images acquired from the same perspective; a damage analysis unit, configured to input the images of the component to be assessed and the control component from N perspectives into the classification model based on a pre-trained classification model, and obtain the image of the component to be assessed from the N perspectives. In the images of the component to be evaluated and the control component at N viewing angles, the probability that the degree of damage of the component to be evaluated relative to the control component is at each of the preset M damage levels is recorded as the probability of each image in the images of the component to be evaluated at each of the M damage levels, where M is a positive integer; and the value assessment unit is configured to determine the damage level of the component to be evaluated based on the probability of each image in the images of the component to be evaluated at each of the M damage levels, so as to determine the degree of damage of the component to be evaluated.

[0012] In some embodiments, the pre-trained classification model is obtained by training based on a deep neural network; wherein, the operation of the damage analysis unit to train the classification model based on the deep neural network includes: generating training samples by manual labeling and / or simulation, wherein the training samples include images of the component to be assessed and the reference component of the ship from more than one perspective, and a given probability that the degree of damage of the component to be assessed relative to the reference component is within the M damage levels based on the comparison between each image of the component to be assessed and the image of the reference component from the same perspective; recording the given probability that the degree of damage of the component to be assessed relative to the reference component is within the M damage levels based on the comparison between each image of the component to be assessed and the image of the reference component from the same perspective as the given probability of each damage level in the images of the component to be assessed from the above perspectives ... in the training samples is within the M damage levels; recording the given probability that the degree of damage of the component to be assessed relative to the reference component in the training samples is within the M damage levels; recording the given probability that the degree of damage of the component to be assessed relative to the reference component in the training samples is within the M damage levels. The images are used as inputs of the deep neural network, and the deep neural network is trained to obtain the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training samples in the M damage levels; the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training samples in the M damage levels is recorded as the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the M damage levels; based on the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the M damage levels, combined with the loss function, the given probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training samples in the M damage levels is used as the output of the deep neural network, and the deep neural network is de-trained to obtain the deep neural network after training and de-training as the classification model.

[0013] In some embodiments, the deep neural network includes: a twin feature extraction network, a saliency network and a fully connected network; wherein the damage analysis unit uses the images of the component to be assessed and the control component of the ship in the training sample from more than one perspective as the input of the deep neural network, trains the deep neural network, and obtains the training probability of each damage level of the images of the component to be assessed in the training sample from more than one perspective at each of the M damage levels, including: extracting image features based on the images of the component to be assessed and the control component of the ship in the training sample from more than one perspective through the twin feature extraction network, and obtaining two sets of image features of the images of the component to be assessed and the control component of the ship in the training sample from more than one perspective. features; using the saliency network, based on two sets of image features of the images of the component to be estimated and the control component from more than one perspective in the training sample, construct an association of image features from more than one perspective, and obtain a set of aggregated features of the images of the component to be estimated and the control component from more than one perspective in the training sample; using the fully connected network, based on the set of aggregated features of the images of the component to be estimated and the control component from more than one perspective in the training sample, perform full connection processing, and output the probability of each image in the images of the component to be estimated from more than one perspective in the training sample for each of the M damage levels, as the training probability of each image in the images of the component to be estimated from more than one perspective in the training sample for each of the M damage levels.

[0014] In some embodiments, the value assessment unit determines the damage grade of the component to be assessed based on the probability of each damage grade among the M damage grades for each image in the N perspectives of the images of the component to be assessed, so as to determine the degree of damage of the component to be assessed, including: selecting a damage grade with the largest probability as the damage grade of the images of the N perspectives of the component to be assessed based on the probability of each damage grade among the M damage grades for each image in the N perspectives of the images of the component to be assessed, or selecting damage grades corresponding to a group of probabilities with probabilities greater than a set value and taking a weighted average as the damage grade of the images of the N perspectives of the component to be assessed; taking a weighted average of the damage grades of the N images of the component to be assessed based on the damage grade of each image of the component to be assessed in the damage grades of the images of the N perspectives of the component to be assessed to obtain the damage grade of the component to be assessed; and further, determining the value of the component to be assessed based on the damage grade of the component to be assessed and the value of the control component, which is recorded as the observed value of the component to be assessed.

[0015] In accordance with the above-mentioned device, the present invention further provides a terminal comprising: the device for determining the degree of damage of a ship component as described above.

[0016] In accordance with the above method, the present invention further provides a storage medium comprising a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned method for determining the degree of damage of ship components.

[0017] Therefore, the solution of the present invention generates training samples by adopting manual labeling and / or simulation methods, wherein the training samples include multi-perspective images of the component to be assessed and the reference component of the ship, and the probability that the degree of damage of the component to be assessed is at a different damage level relative to the reference component based on the comparison between each image of the component to be assessed and the image of the reference component at the same perspective; based on a deep neural network, the multi-perspective images of the component to be assessed and the reference component of the ship are used as input, and the probability that the degree of damage of the component to be assessed is at a different damage level relative to the reference component is used as output, and training is performed in combination with a loss function to obtain a classification network (i.e., a classification model) for classifying the probability that the degree of damage of the component to be assessed is at a different damage level relative to the reference component; when it is necessary to confirm the observation value of the component to be assessed of the ship relative to the reference component, the multi-perspective images of the component to be assessed and the reference component of the ship are input into the classification network, and a classification model based on the multi-perspective images of the component to be assessed and the reference component of the ship are obtained. The probability that each image of the component to be assessed of the ship is at a different damage level relative to the image of the control component from the same perspective is compared with the probability that the degree of damage of the component to be assessed of the ship is at a different damage level relative to the control component; based on the probability that the degree of damage of the component to be assessed of the ship is at a different damage level relative to the control component, the damage level of the component to be assessed of the ship relative to the control component is determined, and then based on the damage level of the component to be assessed of the ship relative to the control component, the observation value of the component to be assessed of the ship relative to the control component is determined, thereby realizing the value assessment of the component to be assessed of the ship; thus, by using the deep learning method to perform ship observation value assessment (such as the determination or assessment of the degree of damage of ship components), the influence of the subjective intention of the ship appraiser during manual assessment on the assessment results (such as the results of the determination or assessment of the degree of damage of ship components) can be avoided, which is conducive to improving the accuracy of the assessment results (such as the results of the determination or assessment of the degree of damage of ship components).

[0018] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention.

[0019] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic flow chart of an embodiment of a method for determining the damage degree of a ship component according to the present invention;

[0021] Figure 2Schematic diagram of a process of obtaining the classification model based on deep neural network training in an embodiment of the method of the present invention;

[0022] Figure 3 1. A flow chart of an embodiment of the method of the present invention for training a neural network using images of a component to be evaluated and a reference component from one or more perspectives in a training sample as inputs;

[0023] Figure 4 A flow chart of an embodiment of the method of the present invention for evaluating the value of a component to be evaluated based on the probability of each image at each damage level and the value of a reference component;

[0024] Figure 5 Schematic diagram of the structure of an embodiment of a device for determining the damage degree of a ship component of the present invention;

[0025] Figure 6 This is a flow chart of an embodiment of a method for assessing the damage degree of ship components based on multi-view salient feature recognition;

[0026] Figure 7 This is a schematic diagram of the workflow of an embodiment of a classification network in a method for assessing the degree of damage of ship components based on multi-view salient feature recognition;

[0027] Figure 8 The figure is a workflow diagram of an embodiment of a mutual saliency network in a method for assessing the degree of damage of ship components based on multi-view saliency feature recognition.

[0028] In conjunction with the accompanying drawings, the reference numerals in the embodiments of the present invention are as follows:

[0029] 102 - Image acquisition unit; 104 - Damage analysis unit; 106 - Value assessment unit. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] As an important means of transportation, ships' asset values ​​fluctuate significantly with shipping market conditions. In markets such as ship leasing, secondhand ship trading, and ship insurance, a ship's observed value is a key factor influencing its asset value and a constant focus for shipowners. To address the issue of assessing the observed value of ship components (e.g., determining or assessing the extent of damage to ship components), ship appraisers use videos and photographs of ship components instead of on-site inspections. However, this approach cannot eliminate the influence of subjective judgment on the assessment results (e.g., the determination or assessment of the extent of damage to ship components), and cannot improve the accuracy of the assessment results. Using image recognition technology to determine damage, a key manifestation of artificial intelligence, can replace manual processing, analysis, and understanding of images with computers to identify various patterns of objects. This allows for rapid and large-scale processing of various physical information. Related solutions have been widely applied in fields such as construction, insurance, healthcare, and meteorology.

[0032] With the development of image recognition technology, methods using computers to automatically recognize images have also been applied to the assessment of the observation value of ship components. However, these methods still have the following drawbacks: they are based solely on images of the ship components to be assessed, lacking comparison with control components of the ship to be assessed. This makes the image recognition task difficult to learn. For example, for image recognition tasks, "directly learning the target" is more difficult than "learning the changes relative to the target." Therefore, in order to avoid the subjective influence of manual evaluation on the observation value of ships and to quickly and effectively assess the observation value of a large number of ships, the present invention proposes a method for determining the degree of damage of ship components. Specifically, it is a method for assessing the degree of damage of ship components based on multi-view salient feature recognition. This method applies artificial intelligence technology based on image recognition to the professional field of ship observation value and can produce highly accurate and objective assessment results (such as the results of determining or assessing the degree of damage of ship components). In the solution of the present invention, deep learning methods are used to assess the value of ship observations (e.g., determining or evaluating the extent of damage to ship components). This can prevent the subjective influence of ship appraisers during manual assessments on the assessment results (e.g., the determination or evaluation of the extent of damage to ship components), thereby improving the accuracy of the assessment results (e.g., the determination or evaluation of the extent of damage to ship components). Furthermore, the addition of learning "component image comparison" can reduce the learning difficulty of the image recognition task.

[0033] According to an embodiment of the present invention, a method for determining the damage degree of a ship component is provided. Figure 1FIG2 is a flow chart of an embodiment of the method of the present invention. The method for determining the damage degree of a ship component may include steps S110 to S130.

[0034] At step S110, when it is necessary to evaluate the observed value of a component of the vessel to be assessed compared to a control component of a reference vessel, images of the component to be assessed and the control component of the reference vessel are acquired from N perspectives, where N is a positive integer. The reference vessel is a vessel of the same type as the vessel to be assessed, with a known specific value. The control component is the same component of the reference vessel used for comparison with the component to be assessed. The images of the component to be assessed and the control component, taken from the same perspective, are acquired from the same perspective.

[0035] Figure 6 This is a flow chart of an embodiment of a method for assessing the damage degree of ship components based on multi-view salient feature recognition. Figure 6 In the example shown, the pre-processing module is used to acquire images. The acquired images include multi-view images of the reference component and the component to be evaluated. To more comprehensively analyze the appearance of the component to be evaluated, the solution of the present invention utilizes multi-view images. By fusing the recognition results from multiple perspectives, a more accurate damage rating is obtained after fusion. This data-driven approach is combined with market expertise to obtain a more accurate observation value assessment conclusion. Here, there are no restrictions on the number and orientation of the multiple perspectives used when acquiring images. For example, common perspectives may include front, rear, left, right, and overhead perspectives. During shooting, regardless of the perspective, it must be ensured that the same image of any reference component and the component to be evaluated is captured from the same perspective. That is, the shooting angles of the images used for comparison should be consistent. In the solution of the present invention, it is assumed that the number of images captured of the component to be evaluated and the reference component is N, where N is a positive integer.

[0036] The reference vessel is a vessel of the same type as the vessel to which the component to be valued belongs, with a known specific value. It can be either second-hand or new. In practice, different valuation methods are used, but it can be assumed that the value and image of the reference vessel are known and available. Images of the reference component can be taken from the reference vessel.

[0037] The vessel to be assessed is the object being assessed. Images of the components to be assessed can be obtained through on-site photography. If the vessel to which the components to be assessed belong is lost or sunk, the most recent images containing the components to be assessed, which were retained before the accident, can be used as the images of the components to be assessed.

[0038] At step S120, based on a pre-trained classification model, the images of the component to be assessed and the control component from N perspectives are input into the classification model, so that the classification model is used to classify the probability that the degree of damage of the component to be assessed relative to the control component is at each of the preset M damage levels, for each image of the component to be assessed and the control component, compared with the image of the control component from the same perspective, in the images of the component to be assessed and the control component from the N perspectives, so as to obtain the probability that the degree of damage of the component to be assessed relative to the control component is at each of the preset M damage levels, for each image of the component to be assessed and the control component, compared with the image of the control component from the same perspective, which is recorded as the probability of each image of the image of the component to be assessed at each of the M damage levels, where M is a positive integer.

[0039] In some embodiments, the classification model pre-trained in step S120 is obtained by training based on a deep neural network. The specific process of training the classification model based on the deep neural network is described in the following exemplary embodiment.

[0040] The following combination Figure 2 The flowchart of an embodiment of the method of the present invention is a flowchart of obtaining the classification model based on deep neural network training, which further illustrates the specific process of obtaining the classification model based on deep neural network training, including: steps S210 to S230.

[0041] In step S210, training samples are generated using manual annotation and / or simulation. The training samples include images of the component to be assessed and a reference component of the ship from one or more viewing angles, and given probabilities that the damage level of the component to be assessed relative to the reference component is within one of the M damage levels, based on a comparison between each image of the component to be assessed and an image of the reference component from the same viewing angle. The given probabilities that the damage level of the component to be assessed relative to the reference component from the same viewing angle is within one of the M damage levels, based on a comparison between each image of the component to be assessed and an image of the reference component from the same viewing angle, are recorded as the given probabilities for each of the M damage levels for each image of the component to be assessed from the above viewing angles.

[0042] In step S220, images of the component to be assessed and the reference component from one or more viewing angles in the training sample are used as input to the deep neural network, and the deep neural network is trained to obtain a training probability for each of the M damage levels for each image in the images of the component to be assessed from the viewing angles in the training sample. The training probability for each of the M damage levels for each image in the images of the component to be assessed from the viewing angles in the training sample is recorded as the training probability for each of the M damage levels for each image in the images of the component to be assessed from the viewing angles.

[0043] In step S230, based on the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the M damage levels, combined with a loss function, the given probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training samples in the M damage levels is used as the output of the deep neural network, and the deep neural network is detrained to obtain the trained and detrained deep neural network as the classification model.

[0044] See also Figure 6 In the example shown, the classification module is used to evaluate and learn the images obtained by the pre-processing module. Before the method of the solution of the present invention is used, the network needs to be trained, and the training adopts a supervised approach. The solution of the present invention uses two methods, manual labeling and simulation, to generate training samples. Among them, manual labeling is that experienced evaluators judge the damage level by comparing the images of the control parts and the parts to be evaluated. However, the amount of manual labeling samples is limited and the acquisition cost is relatively high. Therefore, the solution of the present invention also uses a simulator to model the parts to be evaluated, and simulates the changes in appearance caused by rust, wear, missing, etc. to generate training samples. In the process of evaluating and learning the classification module, reverse training is performed in combination with the loss function to improve the training effect. Among them, the loss function adopts the cross entropy loss function, and the calculation formula is as follows:

[0045]

[0046] Among them, Loss represents the amount of loss. gt represents the true value of the loss level, which is obtained through manual annotation or simulation, that is, the output value of the training sample. f(s) represents the output of the classification module. Each view is a d-dimensional vector. The value of t is as follows:

[0047]

[0048] The purpose of the loss function is to make the probability of the true value level as large as possible. When the probability of the true value gradually increases, the calculation error will gradually decrease, thereby achieving the purpose of network training.

[0049] In some embodiments, the deep neural network includes: a twin feature extraction network, a saliency network, and a fully connected network.

[0050] In step S220, the images of the component to be assessed and the control component of the ship in the training sample from one or more perspectives are used as input to the deep neural network, and the deep neural network is trained to obtain the training probability of each of the M damage levels for each of the images of the component to be assessed in the training sample from the above perspectives. The specific process is described in the following example.

[0051] The following combination Figure 3 The flowchart of an embodiment of the method of the present invention in which images of the component to be evaluated and the reference component in the training sample from one or more perspectives are used as inputs for training the neural network is shown, further illustrating the specific process of using images of the component to be evaluated and the reference component in the training sample from one or more perspectives as inputs for training the neural network in step S220, including: steps S310 to S330.

[0052] In step S310, the twin feature extraction network is used to extract image features based on the images of the component to be estimated and the reference component of the ship in the input training sample from one or more perspectives, thereby obtaining two sets of image features of the images of the component to be estimated and the reference component of the ship in the training sample from one or more perspectives.

[0053] Step S320: Based on the two sets of image features of the image of the component to be estimated and the reference component of the ship in the training sample from one or more perspectives, the saliency network is used to construct an association of image features from one or more perspectives, thereby obtaining a set of aggregated features of the image of the component to be estimated and the reference component of the ship in the training sample from one or more perspectives.

[0054] In step S330, the fully connected network performs fully connected processing based on a set of aggregated features of the images of the component to be assessed and the reference component in the training sample from more than one perspective, and outputs the probability of each damage level in the M damage levels for each image in the images of the component to be assessed from more than one perspective in the training sample as the training probability of each damage level in the M damage levels for each image in the images of the component to be assessed from more than one perspective in the training sample.

[0055] See also Figure 6In the example shown, the classification module uses images of the ship's component to be assessed and the reference component as network inputs, extracts image features using a twin feature extraction network, then associates the two sets of image features using a saliency network. Finally, a fully connected network is used to obtain the rank probability of each view. Specifically, the classification module is used to process multi-view images of the reference component and the component to be assessed using the twin feature extraction network, extracting image features to obtain two sets of images. That is, the twin feature extraction network is used to process the multi-view images of the reference component and extract image features to obtain a set of image features of the reference component. The twin feature extraction network is used to process the multi-view images of the component to be assessed and extract image features to obtain a set of image features of the component to be assessed. The twin feature extraction network is used to extract image features. Furthermore, the classification module is used to associate the two sets of image features (i.e., a set of image features of the reference component and a set of image features of the component to be assessed) using a saliency network. The saliency network is used to construct multi-view feature associations based on the two sets of image features to obtain aggregated features. Finally, the classification module uses a fully connected network to output the probabilities of each level for each view. This allows for classification of the probability of the component under evaluation occurring at different levels of damage relative to the control component. The fully connected network connects every node in the input and output layers and then performs a nonlinear transformation using a nonlinear activation function. Essentially, this transforms one feature space nonlinearly into another, deriving the probabilities of each level for each view based on aggregated features.

[0056] Figure 7 This is a workflow diagram of an embodiment of a classification network in a method for assessing the degree of damage of ship components based on multi-view salient feature recognition. Figure 7 In , cf represents the feature dimension. d represents the damage level of the ship. For example, when d = 3, it means there are 3 damage levels. Figure 7 As shown, Figure 7 As shown, the network input is a set of images from each perspective, with a size of (N, w, h, M), where N represents the number of views, w represents the width of the image, h represents the height of the image, and m represents the number of image channels. For example, in the solution of the present invention, M=3, which represents an RGB color image. The solution of the present invention uses the images in the set to input into a network. Compared with inputting the image of each perspective into a separate network and then fusing them, the solution of the present invention can better fuse the features of each perspective. The feature extraction network maps the image to a high-dimensional feature with a size of (N, w / s, h / s, c), where s is the downsampling size and c is the feature dimension. The twin feature extraction network in this link can use common deep convolutional networks, such as residual networks (ResNet), neural networks (ResNext), etc.

[0057] Furthermore, see Figure 7In the example shown, after obtaining high-dimensional features for both the reference component and the image set of the component to be evaluated, a saliency network is used to establish correlations between these features. Using a saliency network to establish correlations between multiple viewpoints, in layman's terms, images from different viewpoints may share common areas. Establishing correlations between these areas allows for better identification of damaged parts of the component from each viewpoint. The saliency network comprises two submodules: a self-saliency extraction network and a mutual-saliency extraction network. The self-saliency extraction network establishes feature relationships between a pixel in an image and other pixels, while the mutual-saliency extraction network establishes feature relationships between any pixel in the two images. In accordance with the present invention, the self-saliency extraction network can determine which regions in the image of the component to be evaluated are significant and can be used to determine whether the component is damaged. The mutual-saliency extraction network can compare the feature differences between damaged areas in the old and new images of the component to determine the extent of damage. For example, suppose there is a rusty patch on the top of a ship component. A top-down image of the component is taken. Due to the shooting distance, this top-down image includes areas such as the background in addition to the component itself. Using a self-saliency network, the foreground (i.e., the current ship component) can be extracted from the top-down image and areas likely to be rusted can be identified. Once these rusted areas are identified, a mutual saliency network is used to compare them with the same areas in the control component image to determine the extent of damage. For the self-saliency feature network, images from different viewpoints may share common areas. Associating these areas allows for better identification of damaged parts from each viewpoint. The output of both the self-saliency extraction network and the mutual saliency extraction network is a saliency heatmap, which can be compared to assigning a weight to each pixel in the image.

[0058] Figure 8 The figure is a workflow diagram of an embodiment of a mutual saliency network in a method for assessing the degree of damage of ship components based on multi-view saliency feature recognition. Figure 8 As shown, first, the autocorrelation and cross-correlation heat maps are calculated through high-dimensional features, with a size of (N, wh / s^2, wh / s^2), and then the calculated heat maps are used as weights to fuse with high-dimensional features to obtain aggregate features that fuse the properties of each other's images, with a size of (N, w / s, h / s, c f ), where c f Represents the dimension of the feature. Finally, the features of the old and new sets (i.e., the sets of new and old images from each view) are concatenated view-by-view. A max-pooling operation and a fully connected network are used to obtain a feature representation for classification. The size is (N, 1, 1, d), where d represents the number of classes. During use, the d-dimensional vector is normalized to obtain the probability of each class.

[0059] In this solution, images of the vessel component to be appraised and images of a reference component are used as network inputs. By learning changes in image texture, the network identifies the degree of damage to the component's appearance. Compared to related solutions that use computer-generated image recognition methods to assess the component solely based on images of the component, this solution shifts the learning method from texture mapping to texture change mapping, reducing the learning difficulty while increasing the recognition success rate, making the ship valuation process more efficient and accurate.

[0060] The self-saliency network learns texture mapping, while the mutual-saliency network learns texture change mapping. The mutual-saliency network compares old and new images and learns image changes, namely, texture change mapping. Based on this texture change mapping, the network determines the degree of damage to corresponding regions in the new image relative to those in the old image. If only the image of the reference component is used, the image's pixel information is directly converted into damage. In the field of deep learning, it is generally recognized that learning change mapping is easier than directly learning mapping. Therefore, the present invention combines self-saliency network learning with mutual-saliency network learning for model training, reducing learning difficulty while increasing recognition success rate, making the ship valuation process more efficient and accurate.

[0061] In step S130, the damage grade of the component to be assessed is determined based on the probability of each image in the N viewing angles of the component to be assessed being at each of the M damage grades, so as to determine the degree of damage of the component to be assessed.

[0062] The solution of the present invention provides a method for assessing the degree of damage to ship components based on multi-perspective salient feature recognition. It uses deep learning methods to evaluate the observation value of ships (such as determining or evaluating the degree of damage to ship components). Specifically, it uses a deep neural network to evaluate the observation value of the ship's components to be assessed, converting the assessment problem into a damage grade classification problem. By inputting multi-perspective images of the control component and the component to be assessed into the network, features are extracted separately, and the saliency network is used to fuse the features, outputting the damage grade, and finally calculating the observation value based on the damage grade. In this way, based on a deep neural network, through big data training models, no human intervention is required, which greatly improves the automation efficiency of ship component observation value assessment and can reduce assessment costs.

[0063] In some embodiments, in step S130, the damage grade of the component to be assessed is determined based on the probability of each image in the N perspectives of the image of the component to be assessed having each damage grade among the M damage grades, so as to achieve a process of determining the degree of damage of the component to be assessed, as described in the following exemplary embodiment.

[0064] The following combination Figure 4The flowchart of an embodiment of the method of the present invention for evaluating the value of the component to be evaluated based on the probability of each image at each damage level and the value of the reference component is shown, further illustrating the specific process of evaluating the value of the component to be evaluated based on the probability of each image at each damage level and the value of the reference component in step S130, including: steps S410 to S430.

[0065] In step S410, based on the probability of each damage level among the M damage levels for each image in the N viewing angles of the component to be evaluated, the damage level with the largest probability is selected as the damage level of the images of the component to be evaluated from the N viewing angles, or the damage levels corresponding to a group of probabilities with probabilities greater than a set value are selected and a weighted average is taken as the damage level of the images of the component to be evaluated from the N viewing angles.

[0066] Step S420 , based on the damage level of each image of the component to be assessed among the damage levels of the images of the component to be assessed at N viewing angles, a weighted average of the damage levels of the N images of the component to be assessed is taken to obtain the damage level of the component to be assessed.

[0067] Then, in step S430, the value of the component to be estimated is determined based on the depreciation level of the component to be estimated and the value of the reference component, and is recorded as the observed value of the component to be estimated.

[0068] like Figure 6 As shown in Figure 1, when using deep learning methods to evaluate the value of ship observations (such as determining or evaluating the degree of damage to ship components), three modules are used for processing, namely the pre-processing module, the classification module, and the post-processing module. Figure 6 In the example shown, the post-processing module calculates the observed value of a ship component to be assessed. First, the probabilities of each grade for each view output by the classification module are calculated as the damage grade identified in each view. The damage grades identified in each view are then integrated to calculate the damage grade of the entire component to be assessed. Finally, the observed value of the component to be assessed is calculated based on the damage grade of the entire component to be assessed and the value of the reference component. The addition of learning the reference component image reduces the learning difficulty of the image recognition task.

[0069] Specifically, the post-processing module first calculates the depreciation level of each view of the component to be estimated based on the output of the classification module, and then obtains the depreciation level of the entire component to be estimated by fusing the depreciation levels of each view of the component to be estimated. Finally, the observed value of the component to be estimated is calculated based on the depreciation level of the entire component to be estimated and the value of the control component.

[0070] Assuming that the impairment degree is divided into d levels and the output of the classification module has been normalized, the solution of the present invention first selects the k levels with the highest probability for each view of the component to be evaluated, performs weighted averaging, and obtains the impairment level of the view, that is:

[0071]

[0072] Where V represents the impairment level of view i, i represents the number of the image or view in the image set, w represents the probability of the nth highest level of probabilistic impairment, and d represents the specific impairment level. n is the probability corresponding to the nth highest level, d n is the nth highest level. Since the highest k levels are selected, we need to traverse them once. Assuming the highest k = 2 levels are 2 and 3, with corresponding probabilities of 0.3 and 0.4 respectively, then Vi = (2 * 0.3 + 3 * 0.4) / 0.7 = 2.57. After obtaining the loss level of each view, the loss level of the overall component is the weighted average of the levels of each view, that is:

[0073]

[0074] Where t represents the weight of a view. n is the weight of each view. Assuming five views are used, each with equal weight, then t1 = t2 = t3 = t4 = t5 = 0.25. Assuming the value of the reference part is C, and considering only the appearance, the value of the component to be assessed is CV / d. Here, the deterioration level is the score. Consider a brand new reference part with a score of 100 and a value of C. The calculated score of the current part is V. Therefore, the value of the current part is C*(V / d).

[0075] The solution of the present invention provides a method for assessing the degree of deterioration of ship components based on multi-view significant feature recognition. It is a method for automatically assessing the observation value of ship components. It is mainly used for the observation value assessment of ship components in ship value assessment. It can also be applied to the observation value assessment of any object. It can promote the fairness and justice of the relevant market to a certain extent, has a large market value, and is suitable for promotion and use in related fields. Among them, the observation value of ships is assessed by using a deep learning method (such as determining or assessing the degree of deterioration of ship components). Specifically, the evaluation problem is converted into a deterioration grade classification problem by automatically calculating through a neural network, which effectively avoids the disadvantages of large subjective influence and high cost of manual assessment of the observation value of ship components, and significantly improves the accuracy of automated deterioration grade discrimination.

[0076] Using the technical solution of this embodiment, training samples are generated through manual annotation and / or simulation. These training samples include multi-view images of the ship's component to be assessed and a reference component, as well as the probability that the assessed component's damage level is different from that of the reference component, based on the comparison of each image of the assessed component with an image of the reference component from the same viewpoint. Based on a deep neural network, the multi-view images of the ship's component to be assessed and the reference component are used as input, and the probability that the assessed component's damage level is different from that of the reference component is used as output. Training is performed in conjunction with a loss function to obtain a classification network (i.e., a classification model) for classifying the probabilities of the assessed component's damage level being different from that of the reference component. When the observed value of the ship's component to be assessed relative to the reference component is to be verified, the multi-view images of the ship's component to be assessed and the reference component are input into the classification network, and the probability that the assessed component's damage level is different from that of the reference component, based on the comparison of each image of the ship's component to be assessed with an image of the reference component from the same viewpoint, is obtained. Based on the probability that the damage level of the ship's component to be assessed is at different levels relative to the control component, the damage level of the ship's component to be assessed relative to the control component is determined. Furthermore, based on the damage level of the ship's component to be assessed relative to the control component, the observed value of the ship's component to be assessed relative to the control component is determined, thereby achieving a value assessment of the ship's component to be assessed. Thus, by using deep learning methods to assess the observed value of a ship (such as determining or assessing the damage level of a ship component), the influence of the subjective intention of the ship appraiser during manual assessment on the assessment results (such as the results of determining or assessing the damage level of a ship component) can be avoided, which is conducive to improving the accuracy of the assessment results (such as the results of determining or assessing the damage level of a ship component).

[0077] According to an embodiment of the present invention, a device for determining the damage degree of a ship component corresponding to the method for determining the damage degree of a ship component is also provided. Figure 5 The structure diagram of an embodiment of the device of the present invention is shown in FIG. The device for determining the damage degree of a ship component may include: an image acquisition unit 102 , a damage analysis unit 104 , and a value assessment unit 106 .

[0078] The image acquisition unit 102, such as a pre-processing module, is configured to acquire images of the component to be assessed of the vessel to be assessed and the control component of the reference vessel from N perspectives, where N is a positive integer, when it is necessary to evaluate the observed value of the component to be assessed of the vessel to be assessed compared to the control component of the reference vessel. The reference vessel is a vessel of the same type as the vessel to be assessed, with a known specific value. The control component is the same component of the reference vessel used for comparison with the component to be assessed. Among the images of the component to be assessed from N perspectives and the images of the control component from N perspectives, the images of the component to be assessed and the images of the control component used for comparison are a pair of images acquired from the same perspective. The specific functions and processing of the image acquisition unit 102 are described in step S110.

[0079] Figure 6 This is a flow chart of an embodiment of a device for evaluating the observation value of ship components based on multi-view salient feature recognition. Figure 6 In the example shown, the pre-processing module is used to acquire images. The acquired images include images from multiple perspectives of the reference component and the component to be evaluated. To more comprehensively analyze the appearance of the component to be evaluated, the solution of the present invention utilizes multi-perspective images. By fusing the recognition results from multiple perspectives, a more accurate damage rating is obtained after fusion. This data-driven device is combined with market expertise to obtain a more accurate observation value assessment conclusion. The number and orientation of the multiple perspectives used when acquiring images are not restricted. For example, common perspectives may include front, rear, left, right, and overhead perspectives. During shooting, regardless of the perspective, it must be ensured that the same image of any reference component and the component to be evaluated is captured from the same perspective. That is, the shooting angles of the images used for comparison should be consistent. In the solution of the present invention, it is assumed that the number of images captured of the component to be evaluated and the reference component is N, where N is a positive integer.

[0080] The reference vessel is a vessel of the same type as the vessel to which the component to be valued belongs, with a known specific value. It can be either second-hand or new. In actual operation, different valuation devices are used. It can be assumed that the value and image of the reference vessel are both known and available. Images of the reference component can be taken from the reference vessel.

[0081] The vessel to be assessed is the object being assessed. Images of the components to be assessed can be obtained through on-site photography. If the vessel to which the components to be assessed belong is lost or sunk, the most recent images containing the components to be assessed, which were retained before the accident, can be used as the images of the components to be assessed.

[0082] The damage analysis unit 104, such as a classification module, is configured to input images of the component under evaluation and the reference component from N viewing angles into the classification model based on a pre-trained classification model. The classification model is then used to classify the probability of the damage level of the component under evaluation relative to the reference component, as compared to an image of the reference component from the same viewing angle, in the images of the component under evaluation and the reference component from the N viewing angles. This classifies the probability of the damage level of the component under evaluation relative to the reference component from the same viewing angle, as compared to an image of the reference component from the same viewing angle, in the images of the component under evaluation and the reference component from the N viewing angles. The probability of the damage level of the component under evaluation relative to the reference component from the same viewing angle, as compared to an image of the reference component from the same viewing angle, is recorded as the probability of each image of the component under evaluation being in each of the M damage levels, where M is a positive integer. The specific functions and processing of the damage analysis unit 104 are described in step S120.

[0083] In some embodiments, the pre-trained classification model is trained based on a deep neural network. The operation of the loss analysis unit 104 to train the classification model based on the deep neural network is as follows:

[0084] The damage analysis unit 104, such as a classification module, is further configured to generate training samples using manual annotation and / or simulation. The training samples include images of the vessel component to be assessed and a reference component from one or more perspectives, and a given probability that the damage level of the component to be assessed is within one of the M damage levels relative to the reference component, based on a comparison of each image of the component to be assessed with an image of the reference component from the same perspective. The given probability that the damage level of the component to be assessed is within one of the M damage levels, based on a comparison of each image of the component to be assessed with an image of the reference component from the same perspective, is recorded as the given probability of each of the M damage levels for each image of the component to be assessed from the above perspectives. The detailed functions and processing of the damage analysis unit 104 are further described in step S210.

[0085] The damage analysis unit 104, such as a classification module, is further configured to use images of the component to be assessed and the reference component from one or more viewing angles in the training sample as input to the deep neural network, train the deep neural network, and obtain a training probability for each of the M damage levels for each image in the images of the component to be assessed from the above viewing angles in the training sample. The training probability for each of the M damage levels for each image in the images of the component to be assessed from the above viewing angles in the training sample is recorded as the training probability for each of the M damage levels for each image in the images of the component to be assessed from the above viewing angles. The detailed functions and processing of the damage analysis unit 104 are further described in step S220.

[0086] The damage analysis unit 104, such as a classification module, is further configured to use the training probabilities of each of the M damage levels for each image in the images of the component under evaluation from the above perspectives, combined with a loss function, as the output of the deep neural network, perform back-training on the deep neural network, and obtain the trained and back-trained deep neural network as the classification model. The specific functions and processing of the damage analysis unit 104 are further described in step S230.

[0087] See also Figure 6 In the example shown, the classification module is used to evaluate and learn the images obtained by the pre-processing module. Before the device of the solution of the present invention is used, the network needs to be trained, and the training adopts a supervised method. The solution of the present invention uses two methods: manual labeling and simulation to generate training samples. Among them, manual labeling is that experienced evaluators judge the damage level by comparing the images of the control parts and the parts to be evaluated. However, the amount of manual labeling samples is limited and the acquisition cost is relatively high. Therefore, the solution of the present invention also uses a simulator to model the parts to be evaluated, and simulates the changes in appearance caused by rust, wear, missing, etc. to generate training samples. In the process of evaluating and learning the classification module, reverse training is performed in combination with the loss function to improve the training effect. Among them, the loss function adopts the cross entropy loss function, and the calculation formula is as follows:

[0088]

[0089] Among them, Loss represents the amount of loss. gt represents the true value of the loss level, which is obtained through manual annotation or simulation, that is, the output value of the training sample. f(s) represents the output of the classification module. Each view is a d-dimensional vector. The value of t is as follows:

[0090]

[0091] The purpose of the loss function is to make the probability of the true value level as large as possible. When the probability of the true value gradually increases, the calculation error will gradually decrease, thereby achieving the purpose of network training.

[0092] In some embodiments, the deep neural network includes: a twin feature extraction network, a saliency network, and a fully connected network.

[0093] The damage analysis unit 104 uses the images of the component to be assessed and the reference component in the training sample from one or more perspectives as inputs to the deep neural network, trains the deep neural network, and obtains a training probability of each of the images of the component to be assessed in the training sample from the multiple perspectives for each of the M damage levels, including:

[0094] The damage analysis unit 104, such as a classification module, is further configured to extract image features using the twin feature extraction network based on images of the vessel component to be assessed and the reference component in the input training sample from one or more perspectives, thereby obtaining two sets of image features for the images of the vessel component to be assessed and the reference component in the training sample from one or more perspectives. The detailed functions and processing of the damage analysis unit 104 are further described in step S310.

[0095] The damage analysis unit 104, such as a classification module, is further configured to, using the saliency network, establish associations between the image features from one or more perspectives based on the two sets of image features from the images of the vessel component to be assessed and the reference component from one or more perspectives in the training sample, thereby obtaining a set of aggregated features from the images of the vessel component to be assessed and the reference component from one or more perspectives in the training sample. The detailed functions and processing of the damage analysis unit 104 are further described in step S320.

[0096] The damage analysis unit 104, such as a classification module, is further configured to perform fully connected processing using the fully connected network based on a set of aggregated features of images of the component to be assessed and the reference component in the training sample from one or more perspectives, and output a probability of each of the M damage levels for each of the images of the component to be assessed from the above perspectives in the training sample as the training probability of each of the M damage levels for each of the images of the component to be assessed from the above perspectives in the training sample. The specific functions and processing of the damage analysis unit 104 are further described in step S330.

[0097] See also Figure 6In the example shown, the classification module uses images of the ship's component to be assessed and the reference component as network inputs, extracts image features using a twin feature extraction network, then associates the two sets of image features using a saliency network. Finally, a fully connected network is used to obtain the rank probability of each view. Specifically, the classification module is used to process multi-view images of the reference component and the component to be assessed using the twin feature extraction network, extracting image features to obtain two sets of images. That is, the twin feature extraction network is used to process the multi-view images of the reference component and extract image features to obtain a set of image features of the reference component. The twin feature extraction network is used to process the multi-view images of the component to be assessed and extract image features to obtain a set of image features of the component to be assessed. The twin feature extraction network is used to extract image features. Furthermore, the classification module is used to associate the two sets of image features (i.e., a set of image features of the reference component and a set of image features of the component to be assessed) using a saliency network. The saliency network is used to construct multi-view feature associations based on the two sets of image features to obtain aggregated features. Finally, the classification module uses a fully connected network to output the probabilities of each level for each view. This allows for classification of the probability of the component under evaluation occurring at different levels of damage relative to the control component. The fully connected network connects every node in the input and output layers and then performs a nonlinear transformation using a nonlinear activation function. Essentially, this transforms one feature space nonlinearly into another, deriving the probabilities of each level for each view based on aggregated features.

[0098] Figure 7 The following is a workflow diagram of an embodiment of a classification network in a ship component observation value assessment device based on multi-view salient feature recognition. Figure 7 In , cf represents the feature dimension. d represents the damage level of the ship. For example, when d = 3, it means there are 3 damage levels. Figure 7 As shown, Figure 7 As shown, the network input is a set of images from each perspective, with a size of (N, w, h, M), where N represents the number of views, w represents the width of the image, h represents the height of the image, and m represents the number of image channels. For example, in the solution of the present invention, M=3, which represents an RGB color image. The solution of the present invention uses the images in the set to input into a network. Compared with inputting the image of each perspective into a separate network and then fusing them, the solution of the present invention can better fuse the features of each perspective. The feature extraction network maps the image to a high-dimensional feature with a size of (N, w / s, h / s, c), where s is the downsampling size and c is the feature dimension. The twin feature extraction network in this link can use common deep convolutional networks, such as residual networks (ResNet), neural networks (ResNext), etc.

[0099] Furthermore, see Figure 7In the example shown, after obtaining high-dimensional features for both the reference component and the image set of the component to be evaluated, a saliency network is used to establish correlations between these features. Using a saliency network to establish correlations between multiple viewpoints, in layman's terms, images from different viewpoints may share common areas. Establishing correlations between these areas allows for better identification of damaged parts of the component from each viewpoint. The saliency network comprises two submodules: a self-saliency extraction network and a mutual-saliency extraction network. The self-saliency extraction network establishes feature relationships between a pixel in an image and other pixels, while the mutual-saliency extraction network establishes feature relationships between any pixel in the two images. In accordance with the present invention, the self-saliency extraction network can determine which regions in the image of the component to be evaluated are significant and can be used to determine whether the component is damaged. The mutual-saliency extraction network can compare the feature differences between damaged areas in the old and new images of the component to determine the extent of damage. For example, suppose there is a rusty patch on the top of a ship component. A top-down image of the component is taken. Due to the shooting distance, this top-down image includes areas such as the background in addition to the component itself. Using a self-saliency network, the foreground (i.e., the current ship component) can be extracted from the top-down image and areas likely to be rusted can be identified. Once these rusted areas are identified, a mutual saliency network is used to compare them with the same areas in the control component image to determine the extent of damage. For the self-saliency feature network, images from different viewpoints may share common areas. Associating these areas allows for better identification of damaged parts from each viewpoint. The output of both the self-saliency extraction network and the mutual saliency extraction network is a saliency heatmap, which can be compared to assigning a weight to each pixel in the image.

[0100] Figure 8 The figure is a workflow diagram of an embodiment of a mutual saliency network in a ship component observation value assessment device based on multi-view saliency feature recognition. Figure 8 As shown, first, the autocorrelation and cross-correlation heat maps are calculated through high-dimensional features, with a size of (N, wh / s^2, wh / s^2), and then the calculated heat maps are used as weights to fuse with high-dimensional features to obtain aggregate features that fuse the properties of each other's images, with a size of (N, w / s, h / s, c f ), where c f Represents the dimension of the feature. Finally, the features of the new and old sets are concatenated view-by-view, and a max pooling operation and a fully connected network are used to obtain a feature representation for classification. The size is (N, 1, 1, d), where d represents the number of classes. During use, the d-dimensional vector is normalized to obtain the probability of each class.

[0101] In this solution, both images of the vessel component to be appraised and images of a reference component are used as network inputs. By learning changes in image texture, the network identifies the degree of damage to the component's appearance. Compared to related solutions that utilize computer-generated image recognition devices to assess the component based solely on images of the component, this solution shifts the learning method from texture mapping to texture change mapping, reducing the learning difficulty while increasing the recognition success rate, making the ship valuation process more efficient and accurate.

[0102] The self-saliency network learns texture mapping, while the mutual-saliency network learns texture change mapping. The mutual-saliency network compares old and new images and learns image changes, namely, texture change mapping. Based on this texture change mapping, the network determines the degree of damage to corresponding regions in the new image relative to those in the old image. If only the image of the reference component is used, the image's pixel information is directly converted into damage. In the field of deep learning, it is generally recognized that learning change mapping is easier than directly learning mapping. Therefore, the present invention combines self-saliency network learning with mutual-saliency network learning for model training, reducing learning difficulty while increasing recognition success rate, making the ship valuation process more efficient and accurate.

[0103] The value assessment unit 106, such as a post-processing module, is configured to determine the damage level of the component to be assessed based on the probability of each image in the N perspective images of the component to be assessed being in each of the M damage levels, thereby determining the extent of damage to the component to be assessed. The specific functions and processing of the value assessment unit 106 are described in step S130.

[0104] The solution of the present invention provides a device for evaluating the observation value of ship components based on multi-perspective salient feature recognition. It uses a deep learning device to perform ship observation value evaluation (such as determining or evaluating the degree of damage to ship components). Specifically, it uses a deep neural network to evaluate the observation value of the ship's components to be evaluated, converting the evaluation problem into a damage grade classification problem. By inputting multi-perspective images of the control component and the component to be evaluated into the network, extracting features respectively, and using a saliency network to perform feature fusion, outputting the damage grade, and finally calculating the observation value based on the damage grade. In this way, based on a deep neural network, through big data training models, no human intervention is required, which greatly improves the automation efficiency of ship component observation value evaluation and can reduce evaluation costs.

[0105] In some embodiments, the value assessment unit 106 determines the damage level of the component to be assessed based on the probability of each image in the N perspective images of the component to be assessed being at each of the M damage levels, thereby determining the extent of damage of the component to be assessed, including:

[0106] The value assessment unit 106, such as a post-processing module, is further configured to select, based on the probability of each of the M damage levels for each of the N viewing angles of the component to be assessed, the damage level with the highest probability as the damage level for the N viewing angles of the component to be assessed, or to select damage levels corresponding to a set of probabilities greater than a set value and calculate a weighted average thereof as the damage level for the N viewing angles of the component to be assessed. The specific functions and processing of the value assessment unit 106 are further described in step S410.

[0107] The value assessment unit 106, such as a post-processing module, is further configured to calculate a weighted average of the damage levels of the N images of the component to be assessed, based on the damage level of each image of the component to be assessed from the N perspectives, to determine the damage level of the component to be assessed. The specific functions and processing of the value assessment unit 106 are described in step S420.

[0108] Furthermore, the value assessment unit 106, such as a post-processing module, is further configured to determine the value of the component to be assessed based on the depreciation level of the component to be assessed and the value of the reference component, and record this as the observed value of the component to be assessed. The specific functions and processing of the value assessment unit 106 are further described in step S430.

[0109] like Figure 6 As shown in FIG, when using a deep learning device to evaluate the value of ship observation (such as determining or evaluating the degree of damage to ship components), three modules are used for processing, namely, a pre-processing module, a classification module, and a post-processing module. Figure 6 In the example shown, the post-processing module calculates the observed value of a ship component to be assessed. First, the probabilities of each grade for each view output by the classification module are calculated as the damage grade identified in each view. The damage grades identified in each view are then integrated to calculate the damage grade of the entire component to be assessed. Finally, the observed value of the component to be assessed is calculated based on the damage grade of the entire component to be assessed and the value of the reference component. The addition of learning the reference component image reduces the learning difficulty of the image recognition task.

[0110] Specifically, the post-processing module first calculates the depreciation level of each view of the component to be estimated based on the output of the classification module, and then obtains the depreciation level of the entire component to be estimated by fusing the depreciation levels of each view of the component to be estimated. Finally, the observed value of the component to be estimated is calculated based on the depreciation level of the entire component to be estimated and the value of the control component.

[0111] Assuming that the impairment degree is divided into d levels and the output of the classification module has been normalized, the solution of the present invention first selects the k levels with the highest probability for each view of the component to be evaluated, performs weighted averaging, and obtains the impairment level of the view, that is:

[0112]

[0113] Where V represents the impairment level of view i, i represents the number of the image or view in the image set, w represents the probability of the nth highest level of probabilistic impairment, and d represents the specific impairment level. n is the probability corresponding to the nth highest level, d n is the nth highest level. Since the highest k levels are selected, we need to traverse them once. Assuming the highest k = 2 levels are 2 and 3, with corresponding probabilities of 0.3 and 0.4 respectively, then Vi = (2 * 0.3 + 3 * 0.4) / 0.7 = 2.57. After obtaining the loss level of each view, the loss level of the overall component is the weighted average of the levels of each view, that is:

[0114]

[0115] Where t represents the weight of a view. n is the weight of each view. Assuming five views are used, each with equal weight, then t1 = t2 = t3 = t4 = t5 = 0.25. Assuming the value of the reference part is C, and considering only the appearance, the value of the component to be assessed is CV / d. Here, the deterioration level is the score. Consider a brand new reference part with a score of 100 and a value of C. The calculated score of the current part is V. Therefore, the value of the current part is C*(V / d).

[0116] The solution of the present invention provides a device for evaluating the observation value of ship components based on multi-view salient feature recognition. It is a device for automatically evaluating the observation value of ship components. It is mainly used for the observation value evaluation of ship components in ship value evaluation. It can also be used for the observation value evaluation of any object. It can promote the fairness and justice of the relevant market to a certain extent, has a large market value, and is suitable for promotion and use in related fields. Among them, by using a deep learning device to evaluate the observation value of ships (such as determining or evaluating the degree of damage to ship components), specifically, automatically calculating through a neural network, converting the evaluation problem into a damage grade classification problem, effectively avoiding the disadvantages of large subjective influence and high cost of manual evaluation of the observation value of ship components, and significantly improving the accuracy of automatic damage grade discrimination.

[0117] Since the processing and functions implemented by the device of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0118] According to the technical solution of the present invention, a training sample is generated by adopting a manual labeling method and / or a simulation method, wherein the training sample includes multi-perspective images of the component to be assessed and the reference component of the ship, and the probability that the damage degree of the component to be assessed is at a different damage level relative to the reference component based on the comparison between each image of the component to be assessed and the image of the reference component at the same perspective; based on a deep neural network, the multi-perspective images of the component to be assessed and the reference component of the ship are used as input, and the probability that the damage degree of the component to be assessed is at a different damage level relative to the reference component is used as output, and training is performed in combination with a loss function to obtain a classification network (i.e., a classification model) for classifying the probabilities that the damage degree of the component to be assessed is at different damage levels relative to the reference component; when it is necessary to confirm the observed value of the component to be assessed relative to the reference component, the component to be assessed is used as the input. Multi-perspective images of the ship's component to be assessed and the reference component are input into the classification network to obtain the probability that the degree of damage of the ship's component to be assessed is at a different damage grade relative to the reference component based on each image of the ship's component to be assessed compared with the image of the reference component at the same perspective; based on the probability that the degree of damage of the ship's component to be assessed is at a different damage grade relative to the reference component, the damage grade of the ship's component to be assessed relative to the reference component is determined, and then based on the damage grade of the ship's component to be assessed relative to the reference component, the observed value of the ship's component to be assessed relative to the reference component is determined, thereby realizing the value assessment of the ship's component to be assessed, converting the assessment problem into a damage grade classification problem, effectively avoiding the disadvantages of large subjective influence and high cost of manual assessment of the observed value of ship components, and significantly improving the accuracy of automated damage grade discrimination.

[0119] According to an embodiment of the present invention, a terminal corresponding to the device for determining the degree of damage of a ship component is also provided. The terminal may include: the device for determining the degree of damage of a ship component described above.

[0120] Since the processing and functions implemented by the terminal of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned devices, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0121] According to the technical solution of the present invention, a training sample is generated by adopting a manual labeling method and / or a simulation method, wherein the training sample includes multi-perspective images of the component to be assessed and the reference component of the ship, and the probability that the damage degree of the component to be assessed is at a different damage level relative to the reference component based on the comparison between each image of the component to be assessed and the image of the reference component at the same perspective; based on a deep neural network, the multi-perspective images of the component to be assessed and the reference component of the ship are used as input, and the probability that the damage degree of the component to be assessed is at a different damage level relative to the reference component is used as output, and training is performed in combination with a loss function to obtain a classification network (i.e., a classification model) for classifying the probabilities that the damage degree of the component to be assessed is at different damage levels relative to the reference component; when it is necessary to classify the damage degree of the component to be assessed relative to the reference component of the ship, the classification network (i.e., a classification model) is obtained. When the observation value is confirmed, the multi-perspective images of the component to be evaluated and the reference component of the ship are input into the classification network to obtain the probability that the degree of damage of the component to be evaluated relative to the reference component is at a different damage level based on the comparison between each image of the component to be evaluated and the image of the reference component at the same perspective; based on the probability that the degree of damage of the component to be evaluated relative to the reference component is at a different damage level, the damage level of the component to be evaluated relative to the reference component is determined, and then the observation value of the component to be evaluated relative to the reference component is determined based on the damage level of the component to be evaluated relative to the reference component, thereby realizing the value evaluation of the component to be evaluated of the ship without manual intervention, greatly improving the automation efficiency of the observation value evaluation of ship components, and reducing the evaluation cost.

[0122] According to an embodiment of the present invention, a storage medium corresponding to a method for determining the degree of damage of ship components is also provided, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned method for determining the degree of damage of ship components.

[0123] Since the processing and functions implemented by the storage medium of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0124] According to the technical solution of the present invention, training samples are generated by manual labeling and / or simulation, wherein the training samples include multi-perspective images of the component to be assessed and the reference component of the ship, and the probability that the damage degree of the component to be assessed is at a different damage level relative to the reference component based on the comparison between each image of the component to be assessed and the image of the reference component at the same perspective; based on a deep neural network, the multi-perspective images of the component to be assessed and the reference component of the ship are used as input, and the probability that the damage degree of the component to be assessed is at a different damage level relative to the reference component is used as output, and training is performed in combination with a loss function to obtain a classification network (i.e., a classification model) for classifying the probabilities that the damage degree of the component to be assessed is at different damage levels relative to the reference component; when it is necessary to confirm the observed value of the component to be assessed relative to the reference component, the component to be assessed of the ship is compared with the reference component. Multi-view images of the component are input into the classification network to obtain the probability that the damage degree of the component to be assessed is at a different damage level relative to the control component based on each image of the component to be assessed of the ship compared with the image of the control component at the same viewpoint; based on the probability that the damage degree of the component to be assessed of the ship is at a different damage level relative to the control component, the damage level of the component to be assessed of the ship relative to the control component is determined, and then based on the damage level of the component to be assessed of the ship relative to the control component, the observation value of the component to be assessed of the ship relative to the control component is determined, thereby achieving value assessment of the component to be assessed of the ship. This can avoid the influence of the subjective intention of the ship appraiser during manual assessment on the assessment result (such as the determination or assessment result of the damage degree of the ship component), and is conducive to improving the accuracy of the assessment result (such as the determination or assessment result of the damage degree of the ship component).

[0125] In summary, it is easy for those skilled in the art to understand that, under the premise of no conflict, the above-mentioned advantageous methods can be freely combined and superimposed.

[0126] The foregoing description is merely an embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of the claims.

Claims

1. A method for determining the damage degree of a ship component, characterized in that: include: Acquire images of a component to be assessed of the vessel to be assessed and a reference component of the reference vessel from N viewing angles, where N is a positive integer; wherein, among the images of the component to be assessed from the N viewing angles and the images of the reference component from the N viewing angles, the images of the component to be assessed and the images of the reference component used for comparison are a pair of images acquired from the same viewing angle; Based on a pre-trained classification model, images of the component to be assessed and the reference component from N viewing angles are input into the classification model to obtain, for each image of the component to be assessed and the reference component from the N viewing angles, the probability that the damage degree of the component to be assessed relative to the reference component is within each of M preset damage levels, as measured by the probability of each image of the component to be assessed being within each of the M damage levels, where M is a positive integer. The damage grade of the component to be assessed is determined based on the probability of each image in the N viewing angles of the component to be assessed being of each damage grade in the M damage grades, so as to determine the damage degree of the component to be assessed.

2. The method for determining the damage degree of a ship component according to claim 1, characterized in that: The pre-trained classification model is obtained by training based on a deep neural network; wherein the operation of training the classification model based on the deep neural network includes: Training samples are generated using manual annotation and / or simulation, the training samples including images of a component to be assessed and a reference component of the ship from one or more viewing angles, and given probabilities that the damage degree of the component to be assessed relative to the reference component is within the M damage levels based on a comparison between each image of the component to be assessed and an image of the reference component from the same viewing angle. The given probabilities that the damage degree of the component to be assessed relative to the reference component is within the M damage levels based on a comparison between each image of the component to be assessed and an image of the reference component from the same viewing angle are recorded as the given probabilities of each damage level in the images of the component to be assessed from the above viewing angles. Using images of the component to be assessed and the reference component of the ship in the training sample from one or more viewing angles as input to the deep neural network, the deep neural network is trained to obtain a training probability of each damage level among the M damage levels for each image in the images of the component to be assessed from the above viewing angles in the training sample; recording the training probability of each damage level among the M damage levels for each image in the images of the component to be assessed from the above viewing angles in the training sample as the training probability of each damage level among the images of the component to be assessed from the above viewing angles; Based on the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives among the M damage levels, combined with the loss function, the given probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training samples among the M damage levels is used as the output of the deep neural network, and the deep neural network is detrained to obtain the trained and detrained deep neural network as the classification model.

3. The method for determining the damage degree of a ship component according to claim 2, characterized in that: The deep neural network includes: a twin feature extraction network, a saliency network and a fully connected network; The deep neural network is trained using images of the component to be assessed and the reference component of the ship in the training sample from one or more perspectives as input to obtain a training probability of each damage level in the M damage levels for each of the images of the component to be assessed in the training sample from the multiple perspectives, including: Extracting image features based on the input images of the component to be assessed and the reference component of the ship in the training sample from one or more perspectives using the twin feature extraction network to obtain two sets of image features of the component to be assessed and the reference component of the ship in the training sample from one or more perspectives; Using the saliency network, based on two sets of image features of the image of the component to be estimated and the reference component from one or more perspectives in the training sample, an association of the image features from one or more perspectives is constructed to obtain a set of aggregated features of the image of the component to be estimated and the reference component from one or more perspectives in the training sample; The fully connected network performs fully connected processing based on a set of aggregated features of images of the component to be evaluated and the reference component of the ship in the training sample from more than one perspective, and outputs the probability of each damage level in the M damage levels for each image in the images of the component to be evaluated from more than one perspective in the training sample as the training probability of each damage level in the M damage levels for each image in the images of the component to be evaluated from more than one perspective in the training sample.

4. The method for determining the damage degree of a ship component according to any one of claims 1 to 3, characterized in that: Determining the damage grade of the component to be assessed based on the probability of each image in the N viewing angles of the component to be assessed being at each damage grade among the M damage grades, so as to determine the extent of damage of the component to be assessed, includes: Based on the probability of each damage level among the M damage levels for each image of the component to be assessed from the N viewing angles, a damage level with the highest probability is selected as the damage level of the images of the component to be assessed from the N viewing angles, or damage levels corresponding to a group of probabilities with probabilities greater than a set value are selected and a weighted average is taken as the damage level of the images of the component to be assessed from the N viewing angles; Based on the damage grade of each image of the component to be assessed among the damage grades of the images of the component to be assessed from N viewing angles, a weighted average of the damage grades of the N images of the component to be assessed is taken to obtain the damage grade of the component to be assessed. Furthermore, based on the damage grade of the component to be assessed and the value of the control component, the value of the component to be assessed is determined and recorded as the observed value of the component to be assessed.

5. A device for determining the degree of damage of a ship component, characterized in that: include: an image acquisition unit configured to acquire images of a component to be assessed of the vessel to be assessed and a reference component of the reference vessel from N viewing angles, where N is a positive integer; wherein, among the images of the component to be assessed from the N viewing angles and the images of the reference component from the N viewing angles, the image of the component to be assessed and the image of the reference component used for comparison are a pair of images acquired from the same viewing angle; The damage analysis unit is configured to input images of the component to be assessed and the reference component from N viewing angles into a pre-trained classification model, and obtain, for each image of the component to be assessed and the reference component from the N viewing angles, a probability that the damage degree of the component to be assessed relative to the reference component is within each of M preset damage levels, as compared with an image of the reference component from the same viewing angle. The probability is recorded as the probability of each image of the component to be assessed being within each of the M damage levels, where M is a positive integer. The value assessment unit is configured to determine the damage grade of the component to be assessed based on the probability of each damage grade among the M damage grades for each image in the N perspectives of the component to be assessed, so as to determine the degree of damage of the component to be assessed.

6. The device for determining the damage degree of a ship component according to claim 5, characterized in that: The pre-trained classification model is obtained by training based on a deep neural network; wherein the operation of the loss analysis unit to train the classification model based on the deep neural network includes: Training samples are generated using manual annotation and / or simulation, the training samples including images of a component to be assessed and a reference component of the ship from one or more viewing angles, and given probabilities that the damage degree of the component to be assessed relative to the reference component is within the M damage levels based on a comparison between each image of the component to be assessed and an image of the reference component from the same viewing angle. The given probabilities that the damage degree of the component to be assessed relative to the reference component is within the M damage levels based on a comparison between each image of the component to be assessed and an image of the reference component from the same viewing angle are recorded as the given probabilities of each damage level in the images of the component to be assessed from the above viewing angles. Using images of the component to be assessed and the reference component of the ship in the training sample from one or more viewing angles as input to the deep neural network, the deep neural network is trained to obtain a training probability of each damage level among the M damage levels for each image in the images of the component to be assessed from the above viewing angles in the training sample; recording the training probability of each damage level among the M damage levels for each image in the images of the component to be assessed from the above viewing angles in the training sample as the training probability of each damage level among the images of the component to be assessed from the above viewing angles; Based on the training probability of each damage level of each image in the images of the component to be evaluated from the above perspectives among the M damage levels, combined with the loss function, the given probability of each damage level of each image in the images of the component to be evaluated from the above perspectives in the training samples among the M damage levels is used as the output of the deep neural network, and the deep neural network is detrained to obtain the trained and detrained deep neural network as the classification model.

7. The device for determining the damage degree of a ship component according to claim 6, characterized in that: The deep neural network includes: a twin feature extraction network, a saliency network and a fully connected network; The damage analysis unit uses images of the component to be assessed and the reference component of the ship in the training sample from one or more perspectives as input to the deep neural network, trains the deep neural network, and obtains a training probability of each of the images of the component to be assessed in the training sample from the multiple perspectives for each of the M damage levels, including: Extracting image features based on the input images of the component to be assessed and the reference component of the ship in the training sample from one or more perspectives using the twin feature extraction network to obtain two sets of image features of the component to be assessed and the reference component of the ship in the training sample from one or more perspectives; Using the saliency network, based on two sets of image features of the image of the component to be estimated and the reference component from one or more perspectives in the training sample, an association of the image features from one or more perspectives is constructed to obtain a set of aggregated features of the image of the component to be estimated and the reference component from one or more perspectives in the training sample; The fully connected network performs fully connected processing based on a set of aggregated features of images of the component to be evaluated and the reference component of the ship in the training sample from more than one perspective, and outputs the probability of each damage level in the M damage levels for each image in the images of the component to be evaluated from more than one perspective in the training sample as the training probability of each damage level in the M damage levels for each image in the images of the component to be evaluated from more than one perspective in the training sample.

8. The device for determining the damage degree of a ship component according to any one of claims 5 to 7, characterized in that: The value assessment unit determines the damage grade of the component to be assessed based on the probability of each image in the N viewing angles of the component to be assessed being at each of the M damage grades, so as to determine the extent of damage of the component to be assessed, including: Based on the probability of each damage level among the M damage levels for each image of the component to be assessed from the N viewing angles, a damage level with the highest probability is selected as the damage level of the images of the component to be assessed from the N viewing angles, or damage levels corresponding to a group of probabilities with probabilities greater than a set value are selected and a weighted average is taken as the damage level of the images of the component to be assessed from the N viewing angles; Based on the damage grade of each image of the component to be assessed among the damage grades of the images of the component to be assessed from N viewing angles, a weighted average of the damage grades of the N images of the component to be assessed is taken to obtain the damage grade of the component to be assessed. Furthermore, based on the damage grade of the component to be assessed and the value of the control component, the value of the component to be assessed is determined and recorded as the observed value of the component to be assessed.

9. A terminal, characterized in that: include: The device for determining the degree of damage of a ship component according to any one of claims 5 to 8.

10. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the method for determining the degree of damage of a ship component according to any one of claims 1 to 4.

Images