Damage estimation method, system, equipment and medium

Abstract

The embodiment of the invention provides a damage estimation method, system and device and a medium, and the method comprises the steps: constructing a first training sample set which comprises damaged sensor data, a multi-view image sample corresponding to the damaged sensor data, and a text description corresponding to the multi-view image sample; based on the damaged sensor data and the text description, obtaining a predicted multi-view image through a multi-view image generation model; based on the multi-view image sample and the predicted multi-view image, training a multi-view image generation model to obtain a target multi-view image generation model; performing prediction based on the input sensor data and the input text description by using the target multi-view image generation model to obtain a to-be-measured multi-view image; and obtaining the target damage estimation model, and performing prediction based on the to-be-measured multi-view image by using the target damage estimation model to obtain the target damage condition information, so as to improve the damage estimation precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a damage estimation method, system, device and medium. BACKGROUND

[0002] At present, damage estimation as a key technology, its core lies in the precise assessment of the damage of a specific target through various sensor data. This technology has a wide application prospect in many fields such as military, industry, disaster relief, etc. Its development has gone through decades, during which a variety of different types of damage estimation methods have emerged. However, despite the continuous evolution of technology, the existing damage estimation methods still face many challenges, mainly in two aspects: First, the accuracy of damage estimation still needs to be improved. Whether based on artificial experience evaluation or with the help of algorithm automation evaluation, the current accuracy level has not met the demand for high accuracy in actual application, and there is a lot of room for improvement. Second, the lack of visual analysis capability is another key problem. Most of the existing damage estimation methods cannot provide intuitive visual image results, which makes it difficult for evaluators to effectively analyze the confidence of the damage evaluation results, thereby affecting the credibility and application value of the evaluation results. SUMMARY

[0003] The main purpose of the embodiments of the present application is to propose a damage estimation method, system, device and medium, aiming to improve the accuracy of damage estimation.

[0004] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application proposes a damage estimation method, which comprises: constructing a first training sample set, the first training sample set comprising damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples; obtaining a predicted multi-view image based on the damaged sensor data and the text description through a multi-view image generation model; training the multi-view image generation model based on the multi-view image samples and the predicted multi-view image to obtain a target multi-view image generation model; using the target multi-view image generation model to predict based on input sensor data and input text description to obtain a to-be-tested multi-view image; obtaining a target damage estimation model, and using the target damage estimation model to predict based on the to-be-tested multi-view image to obtain target damage information.

[0005] The method provided by the first aspect utilizes a target multi-view image generation model to obtain a to-be-tested multi-view image, and utilizes a target damage estimation model to perform prediction based on the to-be-tested multi-view image, to obtain target damage condition information. The target multi-view image generation model provides a rich data set for the target damage estimation model, which is conducive to improving the generalization capability of the target damage estimation model, thereby improving the damage estimation accuracy, and effectively solving the problem of poor evaluation accuracy of manual estimation and algorithm evaluation in traditional damage evaluation.

[0006] In a possible implementation manner, the obtaining of the target damage estimation model comprises the following steps: A second training sample set is constructed, and the second training sample set comprises a to-be-tested multi-view image output by the target multi-view image generation model and actual damage condition information corresponding to the to-be-tested multi-view image; Based on the to-be-tested multi-view image, predicted damage condition information is obtained by using a damage estimation model; Based on the actual damage condition information and the predicted damage condition information, the damage estimation model is trained to obtain the target damage estimation model.

[0007] In a possible implementation manner, the obtaining of the multi-view image sample corresponding to the damaged sensor data comprises the following steps: The damaged sensor data in a preset time range is obtained by using simulation software; Based on the preset time range, a multi-view image sample corresponding to a middle time point of the preset time range is determined.

[0008] In a possible implementation manner, the multi-view image generation model is a diffusion model, and the diffusion model comprises a text encoder and a sensor data encoder. The obtaining of the predicted multi-view image by using the multi-view image generation model based on the sensor data and the text description comprises: The text encoder is used to encode the text description to obtain text condition features; The sensor data encoder is used to encode the damaged sensor data to obtain sensor data condition features; The diffusion model is used to predict the text condition features and the sensor data condition features to obtain the predicted multi-view image.

[0009] In a possible implementation manner, the training of the multi-view image generation model based on the multi-view image sample and the predicted multi-view image to obtain a target multi-view image generation model comprises: A diffusion loss function is determined based on the multi-view image sample and the predicted multi-view image. The multi-view image generation model is tuned based on the diffusion loss function to obtain the target multi-view image generation model.

[0010] In a possible implementation, the tuning of the multi-view image generation model based on the diffusion loss function to obtain the target multi-view image generation model includes: The first set of parameters of the multi-view image generation model is calculated based on the diffusion loss function to obtain the gradient of each first set of parameters; The gradient of each first set of parameters is optimized and updated by using an Adam optimizer to tune the multi-view image generation model to obtain the target multi-view image generation model.

[0011] In a possible implementation, the training of the damage estimation model based on the actual damage condition information and the predicted damage condition information to obtain the target damage estimation model includes: A loss function is determined based on the actual damage condition information and the predicted damage condition information; The parameters of the damage estimation model are calculated based on the loss function to obtain the gradient of each second set of parameters; The parameter update and learning rate adjustment are performed based on the gradient of the second set of parameters by using an SGD method to tune the damage estimation model to obtain the target damage estimation model.

[0012] To achieve the above object, a second aspect of the embodiments of the present application proposes a damage estimation system, which includes: A sample set construction module is configured to construct a first training sample set, the first training sample set including damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples; A multi-view image prediction module is configured to obtain predicted multi-view images by a multi-view image generation model based on the damaged sensor data and the text descriptions; A target model generation module is configured to train the multi-view image generation model based on the multi-view image samples and the predicted multi-view images to obtain a target multi-view image generation model; A to-be-tested multi-view image acquisition module is configured to obtain to-be-tested multi-view images by the target multi-view image generation model based on input sensor data and input text descriptions; A damage estimation module is configured to obtain a target damage estimation model, and predict target damage condition information by the target damage estimation model based on the to-be-tested multi-view images.

[0013] The system provided by the second aspect uses the target multi-view image generation model to obtain a to-be-tested multi-view image, and uses the target damage estimation model to perform prediction based on the to-be-tested multi-view image, to obtain target damage condition information. The target multi-view image generation model provides a rich data set for the target damage estimation model, which is beneficial to improve the generalization ability of the target damage estimation model, thereby improving the damage estimation accuracy, and effectively solving the problem of poor evaluation accuracy of manual estimation and algorithm evaluation in traditional damage evaluation.

[0014] In a third aspect, an electronic device is provided, which includes a memory and a processor. The memory stores a computer program. The processor implements the damage estimation method according to any possible implementation manner of the first aspect when executing the computer program.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. The computer program, when executed by a processor, implements the damage estimation method according to any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart of a damage estimation method provided by an embodiment of the present application; Figure 2 is a structural block diagram of a diffusion model provided by an embodiment of the present application; Figure 3 is a flowchart of training a multi-view image generation model provided by an embodiment of the present application; Figure 4 is a schematic diagram of a damage estimation system architecture provided by an embodiment of the present application; Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the present specification will be described clearly and completely in conjunction with the drawings in one or more embodiments of the present specification. Obviously, the described one or more embodiments are only a part of the embodiments of the present specification, rather than all the embodiments. Based on one or more embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor should fall within the protection scope of the present document.

[0018] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the description and claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.

[0020] In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as there is no conflict. The embodiments of the application are further described below with reference to the accompanying drawings.

[0021] Figure 1 is an optional flowchart of the damage estimation method provided by the embodiments of the present application, Figure 1 The method in S100-S500 can include but is not limited to the steps.

[0022] In a first aspect, as shown in Figure 1 The embodiments of the present application propose a damage estimation method, which comprises: S100, constructing a first training sample set, the first training sample set comprising damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples.

[0023] It should be noted that the first training sample set is constructed, and the first training sample set comprises damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples, wherein the first training sample set is constructed by simulation software. A large amount of data pairs can be simulated by the simulation software, which can effectively avoid the problems of sample scarcity, high data acquisition cost and difficult annotation in the traditional data collection process, and can flexibly construct complex and diverse scenes and data distribution according to specific needs, accurately cover the data required for model training, and improve the adaptability and generalization performance of the model to complex real scenes.

[0024] S200, obtaining a predicted multi-view image based on the damaged sensor data and the text description through a multi-view image generation model.

[0025] It should be noted that the multi-view image generation model predicts the multi-view image based on damaged sensor data and text description, wherein the damaged sensor data refers to the data collected, transmitted or stored by the sensor due to physical damage, environmental interference or system failure, etc., which is abnormal, distorted or incomplete, and such data cannot accurately reflect the true state of the actual monitoring object; the text description is used to supplement semantic information such as the position and attributes of the target object, which can supplement the lack of data dimension from the conceptual level, and the combination of the damaged sensor data and the text description can break through the limitation of a single modal, so that the model can still capture the essential features of the scene in the case of incomplete data, improve the anti-interference ability of image prediction, and also enhance the logical consistency of the generated content through the semantic guidance of the text, so that the multi-view image is more consistent with the physical laws and semantic logic of the real scene, and provides a more comprehensive visual basis for the training of the subsequent target damage estimation model.

[0026] S300, training a multi-view image generation model based on the multi-view image sample and the predicted multi-view image to obtain a target multi-view image generation model.

[0027] It should be noted that the multi-view image sample is real data, the predicted multi-view image is the output of the multi-view image generation model, and the multi-view image generation model is trained based on the multi-view image sample and the predicted multi-view image to obtain the target multi-view image generation model, which can significantly improve the generalization ability and robustness of the model. The embodiments of the present application learn the real data distribution and the potential generation rule at the same time, so that the model can capture detailed features while mastering more essential semantic structures, thereby showing stronger adaptability and stability in unknown scenes. This training paradigm essentially constructs a bidirectional knowledge channel, which is dynamically balanced through mutual verification, and this dual supervision mechanism effectively avoids the overfitting or underfitting problem caused by a single data source, so that the target multi-view image generation model exhibits more reliable performance in complex and variable actual applications.

[0028] S400, predicting based on the input sensor data and the input text description by using the target multi-view image generation model to obtain a to-be-tested multi-view image.

[0029] S500, obtaining a target damage estimation model, and predicting based on the to-be-tested multi-view image by using the target damage estimation model to obtain target damage information.

[0030] It should be noted that the output of the target multi-view image generation model, i.e., the to-be-tested multi-view image, is taken as the input of the target damage estimation model for cascade training, so as to construct a deep feature transmission chain, and the output of the target multi-view image generation model provides a structured input for the target damage estimation model. This feature cascade not only reduces the noise interference in the original data, but also creates a closer task coupling relationship, so that the target damage estimation model can focus on a higher level of reasoning task, and finally form an end-to-end optimization collaborative effect, thereby significantly improving the accuracy and efficiency of the overall system.

[0031] By the method provided by the first aspect, the to-be-tested multi-view image is obtained by using the target multi-view image generation model, and the target damage situation information is obtained by using the target damage estimation model based on the to-be-tested multi-view image. The target multi-view image generation model provides a rich data set for the target damage estimation model, which is beneficial to improve the generalization ability of the target damage estimation model, thereby improving the accuracy of damage estimation, and effectively solving the problem of poor evaluation accuracy of artificial estimation and algorithm evaluation in traditional damage evaluation.

[0032] In a possible implementation manner, the obtaining of the target damage estimation model includes the following steps: constructing a second training sample set, the second training sample set including a to-be-tested multi-view image output based on the target multi-view image generation model and actual damage situation information corresponding to the to-be-tested multi-view image; obtaining predicted damage situation information by using a damage estimation model based on the to-be-tested multi-view image; and training the damage estimation model based on the actual damage situation information and the predicted damage situation information to obtain the target damage estimation model.

[0033] In some embodiments, it should be noted that the obtaining of the target damage estimation model includes constructing a second training sample set, the second training sample set including a to-be-tested multi-view image output based on the multi-view image generation model and actual damage situation information corresponding to the to-be-tested multi-view image. The second training sample set is obtained by using the multi-view image generation model. This hierarchical processing avoids noise interference in the original data, so that the subsequent damage estimation model can directly learn based on a purer feature space, thereby significantly improving the calculation efficiency and representation accuracy. The damage estimation model can focus on more complex task logic construction based on the to-be-tested multi-view image output by the target multi-view image generation model, so that the model system can decompose complex tasks in a modular manner, reduce the overall modeling difficulty, and reduce the risk of misjudgment caused by insufficient feature representation of a single model through cascade verification of multiple models, so that the entire system has more robust reasoning ability when facing complex inputs.

[0034] It should be noted that the target damage estimation model also includes obtaining predicted damage information based on the to-be-tested multi-view image and the damage estimation model; and training the damage estimation model based on the actual damage information and the predicted damage information to obtain the target damage estimation model. Training the damage estimation model based on the actual damage information and the predicted damage information to obtain the target damage estimation model can significantly improve the generalization ability and robustness of the model. The embodiments of the present application learn the real data distribution and the potential generation rule at the same time, so that the model can capture detailed features while mastering more essential semantic structures, thereby showing stronger adaptability and stability in unknown scenarios. This training paradigm essentially constructs a bidirectional knowledge channel, which is dynamically balanced through mutual verification. This dual supervision mechanism effectively avoids overfitting or underfitting problems caused by a single data source, so that the target damage estimation model can exhibit more reliable performance in complex and variable actual applications.

[0035] In a possible implementation manner, the obtaining of the multi-view image sample corresponding to the damaged sensor data includes the following steps: obtaining the damaged sensor data in a preset time range by using simulation software; and determining the multi-view image sample corresponding to the middle time of the preset time range based on the preset time range.

[0036] It should be noted that obtaining the sensor data in the preset time range by using the simulation software is conducive to constructing a controllable, efficient and highly generalizable data generation system. The simulation software can construct a parameterized virtual scene based on a physical engine, a mathematical model or domain knowledge, generate a large amount of labeled training data by accurately controlling environmental variables, and strictly match the data distribution to the model training requirements to avoid sample deviation caused by uncontrollable environment during real data collection, thereby providing a large number of data pairs meeting the requirements for model training. Secondly, based on the obtained sensor data in the preset time range, the multi-view image sample corresponding to the middle time of the preset time range is determined. The multi-view image sample can be an image sample of more than four views, which is conducive to improving the damage estimation accuracy. In addition, it can be understood that the number of views of the multi-view image sample is not limited in the embodiments of the present application.

[0037] In a possible implementation manner, as Figure 2As shown, the multi-view image generation model is a diffusion model, and the diffusion model includes a text encoder and a sensor data encoder; and the obtaining, by the multi-view image generation model, of the predicted multi-view image based on the sensor data and the text description includes: encoding, by the text encoder, based on the text description to obtain text condition features; encoding, by the sensor data encoder, based on the damaged sensor data to obtain sensor data condition features; and predicting, by the diffusion model, based on the text condition features and the sensor data condition features to obtain the predicted multi-view image.

[0038] The multi-view image generation model is a diffusion model, and the diffusion model includes a text encoder and a sensor data encoder. The text encoder can encode the text description to obtain text condition features, and the sensor data encoder can encode the sensor data to obtain sensor data condition features. The diffusion model is used to predict a predicted multi-view image based on the text condition features and the sensor data condition features. The text encoder and the sensor data encoder form a multi-modal encoder architecture, which provides a bidirectional constraint capability for the diffusion model. The text condition features output by the text encoder can be used as a conditional signal to regulate the denoising direction in the diffusion process, so as to ensure that the generated result conforms to the logic of the language description. The underlying features extracted by the sensor data encoder can retain the physical characteristics and distribution rules of the real data, so that the diffusion model can capture both the visual details of the data and the semantic rules defined by the text, thereby solving the limitations of single modal in feature representation and having stronger scene understanding capability in processing complex tasks.

[0039] Specifically, for step S300, as shown in Figure 3 As shown, the multi-view image generation model is a diffusion model, and the diffusion model includes a text encoder and a sensor data encoder; and the obtaining, by the multi-view image generation model, of the predicted multi-view image based on the sensor data and the text description includes: encoding, by the text encoder, based on the text description to obtain text condition features; encoding, by the sensor data encoder, based on the damaged sensor data to obtain sensor data condition features; and predicting, by the diffusion model, based on the text condition features and the sensor data condition features to obtain the predicted multi-view image.

[0040] S301, determining a diffusion loss function based on the multi-view image sample and the predicted multi-view image.

[0041] S302, parameter tuning of the multi-view image generation model based on the diffusion loss function to obtain a target multi-view image generation model.

[0042] It should be noted that in some embodiments, the process of training the multi-view image generation model includes determining a diffusion loss function based on the multi-view image sample and the predicted multi-view image, adjusting the multi-view image generation model based on the diffusion loss function, and obtaining a target multi-view image generation model. The loss function of the diffusion model is essentially a quantitative indicator of the difference between the data distribution and the generated distribution, and its mathematical form directly determines the bias correction ability of the model in the inverse denoising process. The diffusion loss function can constrain the denoising error under different noise levels, forcing the model to learn multi-level feature representation from rough semantics to fine texture, avoiding the problem of detail blur or pattern collapse caused by single loss constraint of traditional generation model. The parameter adjustment process can promote the model to learn more robust latent representation, so that it can still generate logical results when facing out-of-distribution data.

[0043] In one possible implementation, the adjusting the multi-view image generation model based on the diffusion loss function to obtain the target multi-view image generation model includes calculating a first set of parameters of the multi-view image generation model based on the diffusion loss function to obtain a gradient of each first set of parameters, and using an Adam optimizer to optimize and update the gradient of each first set of parameters to adjust the multi-view image generation model to obtain the target multi-view image generation model.

[0044] It should be noted that the adjusting the multi-view image generation model based on the diffusion loss function to obtain the target multi-view image generation model includes calculating a first set of parameters of the multi-view image generation model based on the diffusion loss function to obtain a gradient of each first set of parameters, and using an Adam optimizer to optimize and update the gradient of each first set of parameters to adjust the multi-view image generation model to obtain the target multi-view image generation model. The diffusion loss function encodes the prior knowledge of the three-dimensional scene into the constraint condition of gradient calculation through the hierarchical denoising process, so that the gradient update of each parameter carries the geometric consistency information across the views. The Adam optimizer ensures that these gradients containing multi-view constraints can be efficiently converted into parameter updates through adaptive gradient scaling and momentum accumulation, which can realize progressive optimization from coarse-grained scene layout to fine-grained texture mapping, and improve the convergence speed and robustness of the parameters in the training process.

[0045] In a possible implementation manner, the target damage estimation model is obtained by training the damage estimation model based on the actual damage condition information and the predicted damage condition information, including: determining a loss function based on the actual damage condition information and the predicted damage condition information; calculating parameters of the damage estimation model based on the loss function to obtain a gradient of each second group of parameters; and performing parameter updating and learning rate adjustment based on the gradient of the second group of parameters by using an SGD method to adjust parameters of the damage estimation model to obtain the target damage estimation model.

[0046] It should be noted that the target damage estimation model is obtained by training the damage estimation model based on the actual damage condition information and the predicted damage condition information, including: determining a loss function based on the actual damage condition information and the predicted damage condition information; calculating parameters of the damage estimation model based on the loss function to obtain a gradient of each second group of parameters; and performing parameter updating and learning rate adjustment based on the gradient of the second group of parameters by using an SGD method to adjust parameters of the damage estimation model to obtain the target damage estimation model. The gradient of each second group of parameters is calculated based on the loss function, which can more accurately reflect the key information contained in the data, so that the parameter updating direction directly points to improving the prediction ability of the model for the target sample. Compared with the characteristic that the parameters can be updated only by traversing the full amount of data in batch gradient descent, the SGD only uses a single or small batch of samples to calculate the gradient each time. This randomness makes the parameter updating path present a “shaking” characteristic in the parameter space, so as to effectively jump out of the local minimum value trap. The random disturbance effect of the SGD is equivalent to injecting a regularization term into the optimization process, which reduces the risk of overfitting of the model to the training samples.

[0047] In a second aspect, an embodiment of the present application provides a damage estimation system, as shown in the accompanying drawings, the system comprises: Figure 4 As shown in the accompanying drawings, the system comprises: A sample set construction module is configured to construct a first training sample set, the first training sample set comprising damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples.

[0048] It should be noted that the first training sample set is constructed, and the first training sample set comprises damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples. The first training sample set is constructed by using simulation software. A large amount of data pairs can be simulated by using the simulation software, which can effectively avoid problems such as sample scarcity, high data acquisition cost, and high labeling difficulty in a traditional data acquisition process, and can flexibly construct complex and diverse scenes and data distribution according to specific requirements, accurately cover data required for model training, and improve the adaptability and generalization performance of the model to complex real scenes.

[0049] Multi-view image prediction module: used for obtaining a predicted multi-view image through a multi-view image generation model based on the damaged sensor data and the text description.

[0050] It should be noted that the multi-view image generation model predicts a multi-view image based on damaged sensor data and a text description, wherein the damaged sensor data refers to a situation that data collected, transmitted or stored by a sensor is abnormal, distorted or incomplete due to factors such as physical damage, environmental interference or system failure, and such data cannot accurately reflect the true state of the actual monitoring object; the text description is used to supplement semantic information such as the position and attributes of the target object, and can supplement the lack of data dimensions from the conceptual level. The combination of the damaged sensor data and the text description can break through the limitations of a single modality, so that the model can still capture the essential features of the scene in the case of incomplete data, improve the anti-interference ability of image prediction, and also enhance the logical consistency of the generated content through the semantic guidance of the text, so that the multi-view image is more consistent with the physical laws and semantic logic of the real scene, and provides a more comprehensive visual basis for the training of the subsequent target damage estimation model.

[0051] Target model generation module: used for training the multi-view image generation model based on the multi-view image sample and the predicted multi-view image, to obtain a target multi-view image generation model.

[0052] It should be noted that the multi-view image sample is real data, and the predicted multi-view image is the output of the multi-view image generation model. Based on the multi-view image sample and the predicted multi-view image, the multi-view image generation model is trained to obtain a target multi-view image generation model, which can significantly improve the generalization ability and robustness of the model. The embodiments of the present application learn the real data distribution and the potential generation rule at the same time, so that the model can capture detailed features while mastering more essential semantic structures, thereby showing stronger adaptability and stability in unknown scenes. This training paradigm essentially constructs a bidirectional knowledge channel, which is dynamically balanced through mutual verification. This dual supervision mechanism effectively avoids the overfitting or underfitting problem caused by a single data source, so that the target multi-view image generation model exhibits more reliable performance in complex and variable actual applications.

[0053] To-be-tested multi-view image acquisition module: used for predicting a to-be-tested multi-view image based on input sensor data and input text description by using the target multi-view image generation model.

[0054] Damage estimation module: used for obtaining a target damage estimation model, and predicting a target damage situation information based on the to-be-tested multi-view image by using the target damage estimation model.

[0055] It should be noted that the output of the target multi-view image generation model, i.e., the to-be-tested multi-view image, is taken as the input of the target damage estimation model for cascade training, so as to construct a deep feature transmission chain, and the output of the target multi-view image generation model provides a structured input for the target damage estimation model. This feature cascade not only reduces the noise interference in the original data, but also creates a closer task coupling relationship, so that the target damage estimation model can focus on a higher level of reasoning task, and finally form an end-to-end optimization synergy effect, thereby significantly improving the accuracy and efficiency of the overall system.

[0056] Through the system provided by the second aspect, the to-be-tested multi-view image is obtained by using the target multi-view image generation model, and the target damage condition information is obtained by using the target damage estimation model to make a prediction based on the to-be-tested multi-view image. The target multi-view image generation model provides a rich data set for the target damage estimation model, which is beneficial to improve the generalization ability of the target damage estimation model, thereby improving the accuracy of damage estimation, and effectively solving the problem of poor evaluation accuracy of artificial estimation and algorithm evaluation in traditional damage evaluation.

[0057] The embodiments of the present application also provide an electronic device, as shown in the figure, the electronic device 1400 includes: Figure 5 The electronic device 1400 includes: one or more processors 1410; a memory 1420 having one or more programs stored thereon, when the one or more programs are executed by the one or more processors 1410, the one or more processors 1410 implement the damage estimation method provided by any one of the embodiments of the present application.

[0058] The memory 1420 as a kind of non-transient network system, it can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory 1420 can include high-speed random access memory, and can also include non-transient memory, such as at least one magnetic disk storage device, flash memory device, or other non-transient solid-state memory device. In some embodiments, the memory 1420 can optionally include a memory 1420 remotely arranged relative to the processor 1410, and these remote memories 1420 can be connected to the processor 1410 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0059] The memory 1420 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1420 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1420 and are invoked and executed by the processor 1410 to implement the method of the embodiments of the present application.

[0060] The processor 1410 can be implemented in the form of a general-purpose CPU (central processing unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.

[0061] In some embodiments, the electronic device further comprises: An input / output interface for realizing information input and output; A communication interface for realizing communication interaction between the device and other devices, which can realize communication through a wired manner (for example, USB, network cable, etc.) or a wireless manner (for example, mobile network, WIFI, Bluetooth, etc.); A bus for transmitting information between various components (for example, the processor 1410, the memory 1420, the input / output interface, and the communication interface) of the device; The processor 1410, the memory 1420, the input / output interface, and the communication interface can realize communication connection between each other within the device through the bus.

[0062] An embodiment of the present application further provides a computer readable storage medium storing computer executable instructions, the computer executable instructions being used to execute the damage estimation method provided by any one of the embodiments of the present application.

[0063] An embodiment of the present application further provides a computer program product, including a computer program or computer instructions, the computer program or computer instructions being stored in a computer readable storage medium, a processor of a computer device reading the computer program or computer instructions from the computer readable storage medium, and the processor executing the computer program or computer instructions to make the computer device execute the damage estimation method provided by any one of the embodiments of the present application.

[0064] The system architecture and application scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of system architecture and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0065] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. In the embodiments of the present application, any reference to memory, storage, database or other medium can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not as a limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0066] As will be appreciated by one of ordinary skill in the art, all or some steps, systems of the above-disclosed methods can be implemented as software, firmware, hardware, or any suitable combination thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0067] The above description illustrates some embodiments of the present application by reference to the accompanying drawings which are described herein for illustrative purposes, and not for limiting the scope of the present application. Any modification, equivalent replacement and improvement made by those of ordinary skill in the art without departing from the scope and spirit of the present application shall fall within the scope of the present application.

[0068] As will be appreciated by one of ordinary skill in the art, all or some steps, systems, functional modules / units of the above-disclosed methods can be implemented as software, firmware, hardware, or any suitable combination thereof.

[0069] The terms "first", "second", "third", "fourth" etc. (if any) in the description and the drawings of the present application are used to distinguish similar objects, and do not necessarily indicate a particular order or sequence. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the present application described herein can be carried out in other than the order shown or described herein. Furthermore, the terms "comprise" and "have", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that comprises a list of steps or units is not necessarily limited to those steps or units that are clearly listed, but can include other steps or units that are not clearly listed or inherent to such processes, methods, products or apparatus.

[0070] In addition, the various functional units in the embodiments of the present application can be integrated in one processing unit, or each can exist as an independent physical unit, or two or more than two of them are integrated in one physical unit. The above-mentioned integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0071] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0072] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or necessary.

[0073] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the right of the embodiments of the present application is not limited thereto. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the right of the embodiments of the present application.

Claims

1. A damage estimation method, characterized in that: The method comprises: Constructing a first training sample set, the first training sample set including damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples; Based on the damaged sensor data and the text description, obtaining a predicted multi-view image through a multi-view image generation model; Based on the multi-view image samples and the predicted multi-view image, training the multi-view image generation model to obtain a target multi-view image generation model; Using the target multi-view image generation model to make predictions based on input sensor data and input text descriptions to obtain a multi-view image to be tested; A target damage estimation model is obtained, and prediction is performed based on the multi-view images to be measured using the target damage estimation model to obtain target damage information.

2. The method according to claim 1, characterized in that The step of obtaining a target damage estimation model comprises the following steps: Constructing a second training sample set, the second training sample set including a multi-view image to be tested outputted by the target multi-view image generation model, and actual damage information corresponding to the multi-view image to be tested; Based on the multi-view images to be measured, obtaining predicted damage information through a damage estimation model; Based on the actual damage situation information and the predicted damage situation information, the damage estimation model is trained to obtain the target damage estimation model.

3. The method according to claim 1, characterized in that The acquisition of multi-view image samples corresponding to the damaged sensor data comprises the following steps: Using simulation software to obtain damaged sensor data within a preset time range; Based on the preset time range, a multi-view image sample corresponding to a middle moment in the preset time range is determined.

4. The method according to claim 1, wherein The multi-view image generation model is a diffusion model, which includes a text encoder and a sensor data encoder; The step of obtaining a predicted multi-view image by a multi-view image generation model based on the damaged sensor data and the text description includes: Using the text encoder, encoding is performed based on the text description to obtain text conditional features; Encoding the damaged sensor data using the sensor data encoder to obtain a sensor data conditional feature; The diffusion model is used to perform prediction based on the text conditional features and the sensor data conditional features to obtain the predicted multi-view image.

5. The method according to claim 1, wherein Training the multi-view image generation model based on the multi-view image samples and the predicted multi-view image to obtain a target multi-view image generation model includes: determining a diffusion loss function based on the multi-view image samples and the predicted multi-view image; The multi-view image generation model is adjusted based on the diffusion loss function to obtain the target multi-view image generation model.

6. The method according to claim 5, characterized in that The adjusting parameters of the multi-view image generation model based on the diffusion loss function to obtain the target multi-view image generation model includes: Calculating a first set of parameters of the multi-view image generation model based on the diffusion loss function, and obtaining a gradient of each of the first set of parameters; The Adam optimizer is used to optimize and update the gradient of each of the first group of parameters to adjust the parameters of the multi-view image generation model to obtain the target multi-view image generation model.

7. The method according to claim 2, characterized in that Training the damage estimation model based on the actual damage information and the predicted damage information to obtain the target damage estimation model includes: determining a loss function based on the actual damage information and the predicted damage information; Calculating parameters of the damage estimation model based on the loss function to obtain a gradient of each of the second set of parameters; The SGD method is used to perform parameter update and learning rate adjustment based on the gradient of the second set of parameters to adjust the parameters of the damage estimation model to obtain the target damage estimation model.

8. A damage estimation system, characterized in that: The system comprises: A sample set construction module is configured to construct a first training sample set, wherein the first training sample set includes damaged sensor data, multi-view image samples corresponding to the damaged sensor data, and text descriptions corresponding to the multi-view image samples; A multi-view image prediction module is configured to obtain a predicted multi-view image through a multi-view image generation model based on the damaged sensor data and the text description; A target model generation module is configured to train the multi-view image generation model based on the multi-view image samples and the predicted multi-view image to obtain a target multi-view image generation model; The module for acquiring the multi-view image to be tested is used to use the target multi-view image generation model to make predictions based on the input sensor data and the input text description to obtain the multi-view image to be tested; The damage estimation module is used to obtain a target damage estimation model, and use the target damage estimation model to make predictions based on the multi-view images to obtain target damage information.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the damage estimation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the damage estimation method according to any one of claims 1 to 7 is implemented.

Images