Active rescue decision method and device based on knowledge reasoning

Abstract

The application provides a kind of active rescue decision method and device based on knowledge reasoning, wherein the method comprises: obtaining a plurality of disaster state categories in disaster scene image, and the category probability corresponding to each disaster state category;Determine a plurality of disaster entities corresponding to a plurality of disaster state categories from disaster knowledge graph, and respectively determine the similarity of each disaster entity and each entity in disaster knowledge graph;According to the category probability corresponding to each disaster state category and the similarity, determine the target entity from each entity of disaster knowledge graph, and take the entity information of target entity as the rescue decision result.The active rescue decision method and device based on knowledge reasoning provided by the application can determine the rescue decision result from disaster knowledge graph according to disaster scene image, so that decision makers do not need to reserve a large amount of professional knowledge, reduce the labor cost and improve the efficiency of rescue decision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an active rescue decision method and device based on knowledge reasoning. BACKGROUND

[0002] The existing rescue decision is made based on the experience of experts, and the disaster scene is analyzed and judged according to the experience of experts, so as to determine the rescue decision.

[0003] The existing rescue decision made based on the experience of experts is made by artificial decision. The scene is often very complex, and different rescue methods and rescue equipment are usually needed for different scenes, and a large amount of professional experience and background knowledge needs to be artificially reserved. SUMMARY

[0004] The present application provides an active rescue decision method and device based on knowledge reasoning, which solves the technical problem that in the prior art, when making a rescue decision for a disaster scene, a technical expert needs to rely on a large amount of professional experience and background knowledge for judgment, resulting in high labor cost.

[0005] The present application provides an active rescue decision method based on knowledge reasoning, comprising:

[0006] Obtaining a plurality of disaster state categories in a disaster scene image and a category probability corresponding to each disaster state category;

[0007] Determining a plurality of disaster entities corresponding to the plurality of disaster state categories from a disaster knowledge graph, and determining the similarity of each disaster entity to each entity in the disaster knowledge graph;

[0008] According to the category probability corresponding to each disaster state category and the similarity, determining a target entity from each entity of the disaster knowledge graph, and taking the entity information of the target entity as a rescue decision result.

[0009] According to the active rescue decision method based on knowledge reasoning provided by the present application, the plurality of disaster state categories include a plurality of building damage state categories and a plurality of building material categories.

[0010] According to the active rescue decision method based on knowledge reasoning provided by the present application, the obtaining of the plurality of disaster state categories in the disaster scene image and the category probability corresponding to each disaster state category comprises:

[0011] Inputting the disaster scene image into a visual perception model to obtain a plurality of disaster state categories in the disaster scene image output by the visual perception model and a category probability corresponding to each disaster state category;

[0012] The visual perception model is obtained by training an initial visual perception model based on disaster scene image samples and disaster state category labels corresponding to the disaster scene image samples.

[0013] According to the active rescue decision-making method based on knowledge reasoning provided in the present application, in the case that the plurality of disaster state categories in the disaster scene image include a plurality of building damage state categories and a plurality of building material categories, the method further includes:

[0014] The first classifier of the initial visual perception model is trained based on disaster scene image samples and building damage state category labels corresponding to the disaster scene image samples, so as to obtain a loss value of the first classifier.

[0015] The second classifier of the initial visual perception model is trained based on disaster scene image samples and building material category labels corresponding to the disaster scene image samples, so as to obtain a loss value of the second classifier.

[0016] The addition value of the loss value of the first classifier and the loss value of the second classifier is determined, and the visual perception model is determined to converge in the case that the addition value is minimum.

[0017] The visual perception model is obtained by combining the first classifier and the second classifier.

[0018] According to the active rescue decision-making method based on knowledge reasoning provided in the present application, the similarity of each disaster entity to each entity in the disaster knowledge graph is determined respectively, including:

[0019] An entity similarity matrix is constructed according to the similarity of any two entities in the disaster knowledge graph.

[0020] The similarity of each disaster entity to each entity in the disaster knowledge graph is determined from the disaster knowledge graph according to the entity similarity matrix.

[0021] According to the active rescue decision-making method based on knowledge reasoning provided in the present application, the similarity of any two entities in the disaster knowledge graph is determined, including:

[0022] The entity vectors of each entity in the disaster knowledge graph are determined, and the vector distance between each entity vector is determined.

[0023] The similarity of any two entities in the disaster knowledge graph is determined according to the vector distance.

[0024] The present application also provides an active rescue decision-making device based on knowledge reasoning, including:

[0025] The disaster situation state category determination module is configured to acquire a plurality of disaster situation state categories in a disaster situation scene image and a category probability corresponding to each disaster situation state category;

[0026] The similarity determination module is configured to determine a plurality of disaster situation entities corresponding to the plurality of disaster situation state categories from the disaster situation knowledge graph and determine a similarity between each disaster situation entity and each entity in the disaster situation knowledge graph.

[0027] The rescue decision result determination module is configured to determine a target entity from each entity in the disaster situation knowledge graph according to the category probability corresponding to each disaster situation state category and the similarity, and take entity information of the target entity as a rescue decision result.

[0028] The present application also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the knowledge reasoning-based active rescue decision method according to any one of the above when executing the computer program.

[0029] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the knowledge reasoning-based active rescue decision method according to any one of the above.

[0030] The present application also provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the knowledge reasoning-based active rescue decision method according to any one of the above.

[0031] The knowledge reasoning-based active rescue decision method and device provided by the present application can actively determine a rescue decision result according to a disaster situation scene image, so that decision makers do not need to reserve a large amount of professional knowledge, the efficiency of rescue decision is improved while the human cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly describe the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative effort.

[0033] Figure 1 is a flowchart of the knowledge reasoning-based active rescue decision method provided by the present application;

[0034] Figure 2 is a flowchart of the active rescue decision-making method based on knowledge reasoning provided by the application;

[0035] Figure 3 is a schematic diagram of the device structure of the active rescue decision-making method based on knowledge reasoning provided by the application;

[0036] Figure 4 is a schematic diagram of the structure of the active rescue decision-making device based on knowledge reasoning provided by the application;

[0037] Figure 5 is a schematic diagram of the structure of the electronic device provided by the application. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0039] Figure 1 is a flowchart of the active rescue decision-making method based on knowledge reasoning provided by the application. Referring to Figure 1 , the active rescue decision-making method based on knowledge reasoning provided by the application can include:

[0040] Step 110, obtaining a plurality of disaster state categories in a disaster scene image and a category probability corresponding to each disaster state category;

[0041] Step 120, determining a plurality of disaster entities corresponding to the plurality of disaster state categories from a disaster knowledge graph, and respectively determining the similarity between each disaster entity and each entity in the disaster knowledge graph;

[0042] Step 130, determining a target entity from each entity of the disaster knowledge graph according to the category probability corresponding to each disaster state category and the similarity, and taking the entity information of the target entity as a rescue decision-making result.

[0043] The execution subject of the active rescue decision method based on knowledge reasoning provided by the application can be an electronic device, a component in the electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the non-mobile electronic device can be a server, a network attached storage (NAS), or a personal computer (PC), and the application is not limited in this regard.

[0044] The technical solution of the application will be described in detail below with the computer executing the active rescue decision method based on knowledge reasoning provided by the application as an example.

[0045] In step 110, after the disaster occurs, a disaster scene image is acquired, and according to the disaster scene image, a plurality of disaster state categories in the disaster scene image and a category probability corresponding to each disaster state category are acquired.

[0046] The disaster scene image refers to a disaster state image of a disaster scene after a disaster occurs. The disaster can include natural disasters such as earthquake disasters and tsunami disasters, or man-made disasters such as house collapse and building damage.

[0047] The disaster state category is used to reflect the disaster situation in the disaster scene image. The disaster state category can include a building damage state category and a building material category. Through analysis of the disaster state category, the disaster situation in the disaster scene can be determined, so that the rescue decision to be taken can be determined.

[0048] Optionally, the disaster state category in the disaster scene image can be classified by using the trained classifier. Thus, the plurality of disaster state categories in the disaster scene image and the category probability corresponding to each disaster state category are determined.

[0049] In step 120, a disaster knowledge graph is constructed, and a plurality of disaster entities corresponding to the plurality of disaster state categories are determined from the pre-constructed disaster knowledge graph. The similarity between each disaster entity in the disaster entity and each entity in the disaster knowledge graph is calculated to determine the similarity between the disaster entity and each entity in the disaster knowledge graph.

[0050] Knowledge Graph, also known as knowledge domain visualization or knowledge field mapping map in the library and information field, is a series of various graphs showing the development process and structural relationship of knowledge. It uses visualization technology to describe knowledge resources and their carriers, and excavates, analyzes, constructs, draws and displays knowledge and their mutual relationships.

[0051] The disaster condition knowledge graph is determined based on all disaster condition state categories that can occur in the disaster condition scene and rescue strategies corresponding to each disaster condition state category. By constructing entities of each disaster condition state category and relationships corresponding to each entity, a disaster condition knowledge graph of disaster condition knowledge reasoning relationships is determined. The rescue strategies include rescue measures, rescue methods and rescue equipment.

[0052] It can be understood that after determining the disaster condition state categories in the disaster condition scene, since the disaster condition knowledge graph is constructed based on the disaster condition state categories, the entities corresponding to the disaster condition state categories in the disaster condition knowledge graph can be determined. According to the entities corresponding to the disaster condition state categories in the disaster condition knowledge graph, the entities related to the rescue measures, rescue methods and rescue equipment of the entities corresponding to each disaster condition state category can be determined from the disaster condition knowledge graph, so as to finally realize the determination of the rescue decision in the disaster condition scene in the manner of knowledge reasoning according to the disaster condition knowledge graph.

[0053] In step 130, according to the category probability corresponding to each disaster condition state category determined in step 110 and the similarity between each disaster condition entity and each entity in the disaster condition knowledge graph determined in step 120, a target entity is determined from each entity in the disaster condition knowledge graph, and the entity information of the target entity is taken as the rescue decision result.

[0054] It can be understood that the category probability corresponding to each disaster condition state category reflects the possibility of each disaster condition state category. For example, if the disaster condition state categories contained in the disaster condition scene image are building tilt, building partial collapse and debris, and the probability of the building partial collapse category in the disaster condition state categories is the largest, then for the disaster condition scene image, it indicates that the possibility of building collapse is the largest.

[0055] Since the disaster condition knowledge graph is determined based on all disaster condition state categories that can occur in the disaster condition scene and rescue strategies corresponding to each disaster condition state category, the target entity can be determined from the disaster condition knowledge graph according to each disaster condition state category and the similarity between each disaster condition entity and each entity in the disaster condition knowledge graph, and the entity information of the target entity is taken as the rescue decision result in the disaster condition scene image.

[0056] Optionally, since the rescue strategy includes rescue measures, rescue methods and rescue equipment, the finally determined target entity can include multiple. Multiple entities can be determined from the disaster knowledge graph to finally determine the rescue decision result, that is, to determine the entities corresponding to the rescue measures, the entities corresponding to the rescue methods and the entities corresponding to the rescue equipment, and the entity information of the determined multiple target entities is taken as the rescue decision result.

[0057] Optionally, the multiple disaster state categories in the disaster scene image include multiple building damage state categories and multiple building material categories, and determining the rescue decision result can include:

[0058] For the input disaster scene image x i , the feature extractor f aims to extract the global feature description f(x i ) corresponding to the image x i , and the multiple building damage state categories and the multiple building material categories in the input image and the category probabilities corresponding to each category are obtained through the classifiers g1 and g2. The classifier g1 will predict each category in the building damage state category {1,…,K′} that the image may contain and the probability g1(f(x i )) corresponding to the category according to the global feature description f(x i ), and the classifier g2 will predict each category in the building material category {K′+1,…,K} that the image may contain and the probability g2(f(x i )) corresponding to the category according to the global feature description f(x i ).

[0059] For the image x i , the finally determined rescue decision value C(x i ) is:

[0060]

[0061] Where j is a disaster state category, including multiple building damage state categories and multiple building material categories; g1(f(x i )) j is the category probability of the building damage state category of the disaster state category j, g2(f(x i )) j is the category probability of the building material category of the disaster state category j; map(j) is the disaster entity corresponding to the category j in the disaster knowledge graph, U is a part of the entity set in all entities in the disaster knowledge graph set according to the need, u is any entity in the entity set U, and A(map(j),u) is the similarity between any entity u in U and the disaster entity corresponding to the category j in the disaster knowledge graph.

[0062] Specifically, the entities of the disaster situation knowledge graph included in U can be set as required. For example, if it is determined that U is an entity related to all rescue equipment in the disaster situation knowledge graph, all rescue equipment that meets the requirements can be determined from the set U.

[0063] determining the part entity u that meets the requirement of the rescue decision value C(x i ) as the target entity, so as to realize the determination of the target entity. For example, after determining the rescue decision value C(x i ) of all entities in the entity set U, all determined rescue decision values C(x i ) can be sorted from large to small, and the top five rescue decision values C(x i ) corresponding to the entities u are selected as the target entities.

[0064] The active rescue decision method based on knowledge reasoning provided by the embodiment of the application determines the target entity from the disaster situation knowledge graph as the rescue decision result according to the category probability corresponding to each disaster situation category and the similarity between each disaster situation category and each entity in the disaster situation knowledge graph, by acquiring a plurality of disaster situation categories in a disaster situation image, which realizes active determination of the rescue decision result according to the disaster situation image, so that the decision maker does not need to reserve a large amount of professional knowledge, and the efficiency of the rescue decision is improved while the human cost is reduced.

[0065] In one embodiment, the plurality of disaster situation categories includes a plurality of building damage state categories and a plurality of building material categories.

[0066] For a disaster situation, the building damage state can directly reflect the severity of the disaster situation. The building damage state category can include building tilting, building partial collapse, debris, etc. For a disaster situation, different materials of buildings require different materials of rescue tools and rescue methods, and the corresponding rescue strategy for the current disaster situation can be determined by the building material. The building material category can include wood, stone, brick and reinforced concrete, etc.

[0067] It can be understood that by determining the plurality of building damage state categories and the plurality of building material categories in the disaster situation image, the determination of the building damage situation in the disaster situation can be realized, and the corresponding rescue decision result can be determined according to the damage situation and the building material.

[0068] The active rescue decision method based on knowledge reasoning provided by the embodiment of the present application determines multiple disaster condition state categories in a disaster condition scene image, including multiple building damage state categories and multiple building material categories, determines the damage state of the disaster condition scene according to the building damage state, and further determines the rescue decision according to the building material category. The building damage state category and the building material category are used to accurately determine the rescue decision.

[0069] In one embodiment, the multiple disaster condition state categories in the disaster condition scene image and the category probabilities corresponding to the disaster condition state categories are obtained by inputting the disaster condition scene image into a visual perception model to obtain the multiple disaster condition state categories in the disaster condition scene image output by the visual perception model and the category probabilities corresponding to the disaster condition state categories. The visual perception model is obtained by training the visual perception model based on disaster condition scene image samples and disaster condition state category labels marked in the disaster condition scene image samples.

[0070] The disaster condition scene image is input into the visual perception model, and the disaster condition state categories in the disaster condition scene image are identified according to the visual perception model to obtain the multiple disaster condition state categories in the disaster condition scene image output by the visual perception model and the category probabilities corresponding to the disaster condition state categories.

[0071] The visual perception model is a supervised model, and the visual perception model is obtained by training an initial visual perception model based on disaster condition scene image samples and disaster condition state category labels marked in the disaster condition scene image samples.

[0072] Optionally, the visual perception model can be a trained classifier, and the classification of the disaster condition state categories in the disaster condition scene image is realized by the classifier to determine the multiple disaster condition state categories in the disaster condition scene image and the category probabilities corresponding to the disaster condition state categories.

[0073] The active rescue decision method based on knowledge reasoning provided by the embodiment of the present application determines multiple disaster condition state categories in a disaster condition scene image, including multiple building damage state categories and multiple building material categories, determines the damage state of the disaster condition scene according to the building damage state, and further determines the rescue decision according to the building material category. The building damage state category and the building material category are used to accurately determine the rescue decision.

[0074] In an embodiment, in the case that the plurality of disaster condition state categories in the disaster scene image includes a plurality of building damage state categories and a plurality of building material categories, the method further comprises: training a first classifier of the initial visual perception model based on disaster scene image samples and building damage state category labels corresponding to the disaster scene image samples, to obtain a loss value of the first classifier; training a second classifier of the initial visual perception model based on disaster scene image samples and building material category labels corresponding to the disaster scene image samples, to obtain a loss value of the second classifier; determining an addition value of the loss value of the first classifier and the loss value of the second classifier, and determining that the visual perception model converges in the case that the addition value is minimum; and the visual perception model is obtained by combining the first classifier and the second classifier.

[0075] In the case that the plurality of disaster condition state categories in the disaster scene image includes a plurality of building damage state categories and a plurality of building material categories, it is necessary to respectively perform category classification and category probability determination for the building damage state and the building material category.

[0076] The visual perception model is constructed by the first classifier and the second classifier, so as to respectively classify the building damage state and the building material category.

[0077] The visual perception model is composed of a feature extractor f and two classifiers g1 and g2. For an input disaster scene image x i , the feature extractor f aims to extract a global feature description f(x i ) corresponding to the image x i , and obtain a plurality of building damage state categories and a plurality of building material categories and category probabilities corresponding to the categories of the input image through the classifiers g1 and g2. The classifier g1 will predict each category in the building damage state category {1,…,K′} and the probability g1(f(x i )) corresponding to the category according to the global feature description f(x i ), and the classifier g2 will predict each category in the building material category {K′+1,…,K} and the probability g2(f(x i )) corresponding to the category according to the global feature description f(x i ). In order to make the visual perception model have a more accurate prediction result for the image, the addition value of the loss value of the first classifier and the loss value of the second classifier, i.e., the visual perception model loss value L c , is determined as follows:

[0078]

[0079] wherein N is the number of images in the disaster scene image samples, xi is the i-th image of the disaster scene image sample, z ij ∈{0,1}, when the image x i contains the j-th type, z ij =1, otherwise z ij =0, g1(f(x i )) j is the class probability of the building damage state class of the disaster state class j, g2(f(x i )) j is the class probability of the building material class of the disaster state class j.

[0080] In the case that the sum L c of the loss value of the first classifier and the loss value of the second classifier is minimum, it is determined that the visual perception model composed of the first classifier and the second classifier converges.

[0081] The active rescue decision method based on knowledge reasoning provided by the embodiment of the application provides a basis for subsequent implementation of determination of rescue decision results by constructing a visual perception model through a first classifier and a second classifier and training in the case that multiple disaster state classes in a disaster scene image include multiple building damage state classes and multiple building material classes.

[0082] In one embodiment, the similarity of each disaster entity to each entity in the disaster knowledge graph is determined respectively, including: constructing an entity similarity matrix according to the similarity of any two entities in the disaster knowledge graph; and determining the similarity of each disaster entity to each entity in the disaster knowledge graph from the disaster knowledge graph according to the entity similarity matrix.

[0083] For all entities in the disaster knowledge graph, the similarity of any two entities in all entities is calculated, and the entity similarity matrix is constructed according to the similarity of any two entities.

[0084] It can be understood that after the similarity matrix and each disaster entity are determined, the similarity of each disaster entity to each entity in the disaster knowledge graph can be quickly obtained by querying the entity similarity matrix.

[0085] The active rescue decision method based on knowledge reasoning provided by the embodiment of the application constructs an entity similarity matrix through the similarity of any two entities in the disaster knowledge graph, and quickly obtains the similarity of each disaster entity to each entity in the disaster knowledge graph by querying the entity similarity matrix according to the entity similarity matrix.

[0086] In one embodiment, the similarity between any two entities in the disaster knowledge graph is determined, comprising: determining entity vectors of each entity in the disaster knowledge graph according to each entity in the disaster knowledge graph, and determining vector distances between each entity vector; and determining the similarity between any two entities in the disaster knowledge graph according to the vector distances.

[0087] According to each entity in the disaster knowledge graph, the vector representation of each entity is first determined.

[0088] Optionally, for any triple (h, l, t) in the disaster knowledge graph S, the function relationship generated by the edge marked by l corresponds to a mapping transformation, that is, when the triple (h, l, t) is true (t should be the nearest neighbor of h+l), that is, h+l≈t is constrained. Otherwise, h+l should be far away from t. According to the energy-based framework, for some dissimilarity measure d, the energy of the triple is equal to d(h+l, t). In order to learn this mapping transformation, a ranking loss L g is minimized in the training set

[0089]

[0090] S' (h,l,t) = {(h, l, t) | h' ∈ E} ∪ {(h, l, t') | t' ∈ E}

[0091] where (h, l, t) is a triple in the disaster knowledge graph that satisfies the mapping transformation, (h', l, t') is a triple in the disaster knowledge graph that does not satisfy the mapping transformation, S' (h,l,t) is the set of (h', l, t') type triples, [γ+d(h+l, t)-d(h'+l, t')] + represents the positive sample part of γ+d(h+l, t)-d(h'+l, t'), γ>0 is a boundary hyperparameter defined by a person, and E is the set of all entities in the disaster knowledge graph.

[0092] By determining the convergence of the loss function, the relationship of the mapping transformation satisfied between the triples is learned, so as to determine the entity vectors of each entity in the disaster knowledge graph.

[0093] After obtaining the vector representation V(e i ) of each entity e i in the disaster knowledge graph that satisfies the mapping transformation, the similarity between any two entities e i and e j in the disaster knowledge graph can be determined by calculating the vector distance between the entity vectors, and a similarity matrix A(i,j) = sim<V(e i ), V(e j )> is constructed.

[0094] The method for active rescue decision based on knowledge reasoning provided by the embodiments of the present application determines the similarity of any two entities in the disaster knowledge graph through the vector distance between each entity vector in the disaster knowledge graph, and constructs an entity similarity matrix. According to the entity similarity matrix, the similarity of each disaster entity and each entity in the disaster knowledge graph is quickly obtained by querying the entity similarity matrix.

[0095] The following takes a flowchart of an application of the method for active rescue decision based on knowledge reasoning provided by the present application as an example to illustrate the technical solutions provided by the present application: Figure 2 For example, the technical solutions provided by the present application are illustrated:

[0096] In step 210, based on a visual perception model, a plurality of disaster state categories in a disaster scene image and the category probabilities corresponding to each disaster state category are obtained.

[0097] In step 220, the entity vectors corresponding to all entities in the disaster knowledge graph are obtained. Each entity vector satisfies the mapping relationship between the relationship and the entity in the triple.

[0098] In step 230, the similarity of any two entities in the disaster knowledge graph is determined according to the vector distance between each entity vector in the disaster knowledge graph. According to the category probability corresponding to each disaster state category and the similarity of any two entities in the disaster knowledge graph, the best rescue measure entity, rescue method entity and rescue equipment entity in the disaster scene are determined from each entity in the disaster knowledge graph. The entity information of the determined rescue measure entity, rescue method entity and rescue equipment entity is taken as the rescue decision result.

[0099] The following takes a device structure diagram of an application of the method for active rescue decision based on knowledge reasoning provided by the present application as an example to illustrate the technical solutions provided by the present application:

[0100] As shown in the figure, the device can include: Figure 3

[0101] The knowledge graph mapping module 310 is configured to obtain the entity vectors corresponding to all entities in the disaster knowledge graph. The disaster knowledge graph is constructed, and the entity vector of each entity is determined according to each entity in the disaster knowledge graph. Each entity vector satisfies the mapping relationship between the relationship and the entity in the triple.

[0102] The visual perception module 320 is configured to obtain a plurality of building damage state categories and the corresponding category probabilities in the plurality of disaster state categories in the disaster scene image based on the first classifier in the visual perception model. The visual perception module 320 is configured to obtain a plurality of building material categories and the corresponding category probabilities in the plurality of disaster state categories in the disaster scene image based on the second classifier in the visual perception model.

[0103] ​The decision reasoning module 330 is configured to determine the similarity between any two entities in the disaster situation knowledge graph according to the vector distance between the entity vectors of the entities, and construct an entity similarity matrix. According to the category probability corresponding to each disaster situation category and the entity similarity matrix, a plurality of target entities are determined from the entities in the disaster situation knowledge graph, and the entity information of the target entities is taken as the rescue decision result.

[0104] Figure 4 A structure diagram of the active rescue decision device based on knowledge reasoning provided by the application is shown in Figure 4 The device comprises:

[0105] The disaster situation category determination module 410 is configured to obtain a plurality of disaster situation categories in the disaster scene image and the category probability corresponding to each disaster situation category.

[0106] The similarity determination module 420 is configured to determine a plurality of disaster entities corresponding to the plurality of disaster situation categories from the disaster situation knowledge graph, and determine the similarity between each disaster entity and each entity in the disaster situation knowledge graph.

[0107] The rescue decision result determination module 430 is configured to determine target entities from the entities in the disaster situation knowledge graph according to the category probability corresponding to each disaster situation category and the similarity, and take the entity information of the target entities as the rescue decision result.

[0108] The active rescue decision device based on knowledge reasoning provided by the embodiment of the application can determine the target entities from the disaster situation knowledge graph as the rescue decision result by obtaining the plurality of disaster situation categories in the disaster scene image, according to the category probability corresponding to each disaster situation category and the similarity between each disaster situation category and each entity in the disaster situation knowledge graph, which can actively determine the rescue decision result according to the disaster scene image, so that the decision maker does not need to reserve a large amount of professional knowledge, and the efficiency of the rescue decision is improved while the human cost is reduced.

[0109] In one embodiment, the disaster situation category determination module 410 is specifically configured to:

[0110] The plurality of disaster situation categories include a plurality of building damage state categories and a plurality of building material categories.

[0111] In one embodiment, the disaster situation category determination module 410 is specifically configured to:

[0112] The obtaining of the plurality of disaster situation categories in the disaster scene image and the category probability corresponding to each disaster situation category comprises:

[0113] input the disaster situation scene image into a visual perception model to obtain a plurality of disaster situation state categories in the disaster situation scene image output by the visual perception model and a category probability corresponding to each disaster situation state category;

[0114] The visual perception model is obtained by training an initial visual perception model based on disaster situation scene image samples and disaster situation state category labels corresponding to the disaster situation scene image samples.

[0115] In an embodiment, the disaster situation state category determination module 410 is further specifically configured to:

[0116] In the case that the plurality of disaster situation state categories in the disaster situation scene image include a plurality of building damage state categories and a plurality of building material categories, the disaster situation state category determination module 410 is further specifically configured to:

[0117] The first classifier of the initial visual perception model is trained based on disaster situation scene image samples and building damage state category labels corresponding to the disaster situation scene image samples to obtain a loss value of the first classifier.

[0118] The second classifier of the initial visual perception model is trained based on disaster situation scene image samples and building material category labels corresponding to the disaster situation scene image samples to obtain a loss value of the second classifier.

[0119] The addition value of the loss value of the first classifier and the loss value of the second classifier is determined, and the visual perception model is determined to converge in the case that the addition value is minimum.

[0120] The visual perception model is obtained by combining the first classifier and the second classifier.

[0121] In an embodiment, the similarity determination module 420 is specifically configured to:

[0122] The similarity between each disaster entity and each entity in the disaster knowledge graph is determined respectively, including:

[0123] An entity similarity matrix is constructed according to the similarity between any two entities in the disaster knowledge graph.

[0124] The similarity between each disaster entity and each entity in the disaster knowledge graph is determined from the disaster knowledge graph according to the entity similarity matrix.

[0125] In an embodiment, the similarity determination module 420 is further specifically configured to:

[0126] The similarity between any two entities in the disaster knowledge graph is determined, including:

[0127] According to each entity in the disaster situation knowledge graph, an entity vector of each entity is determined, and a vector distance between each entity vector is determined.

[0128] According to the vector distance, a similarity between any two entities in the disaster situation knowledge graph is determined.

[0129] Figure 5 An entity structure diagram of an electronic device is shown, as Figure 5 The electronic device can include a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can invoke a logical instruction in the memory 530 to execute a knowledge reasoning-based active rescue decision method, which includes:

[0130] Obtaining a plurality of disaster situation state categories in a disaster situation scene image, and a category probability corresponding to each disaster situation state category;

[0131] Determining a plurality of disaster entities corresponding to the plurality of disaster situation state categories from a disaster situation knowledge graph, and determining a similarity between each disaster entity and each entity in the disaster situation knowledge graph;

[0132] According to the category probability corresponding to each disaster situation state category and the similarity, a target entity is determined from each entity in the disaster situation knowledge graph, and entity information of the target entity is taken as a rescue decision result.

[0133] In addition, the logical instruction in the memory 530 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0134] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the knowledge reasoning-based active rescue decision method provided by the above method, which comprises:

[0135] obtaining a plurality of disaster state categories in a disaster scene image and a category probability corresponding to each disaster state category;

[0136] determining a plurality of disaster entities corresponding to the plurality of disaster state categories from a disaster knowledge graph, and determining a similarity between each disaster entity and each entity in the disaster knowledge graph;

[0137] determining a target entity from each entity in the disaster knowledge graph according to the category probability corresponding to each disaster state category and the similarity, and taking entity information of the target entity as a rescue decision result.

[0138] In another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the knowledge reasoning-based active rescue decision method provided by the above method, which comprises:

[0139] obtaining a plurality of disaster state categories in a disaster scene image and a category probability corresponding to each disaster state category;

[0140] determining a plurality of disaster entities corresponding to the plurality of disaster state categories from a disaster knowledge graph, and determining a similarity between each disaster entity and each entity in the disaster knowledge graph;

[0141] determining a target entity from each entity in the disaster knowledge graph according to the category probability corresponding to each disaster state category and the similarity, and taking entity information of the target entity as a rescue decision result.

[0142] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0143] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0144] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A proactive rescue decision-making method based on knowledge reasoning, characterized in that, include: Obtain multiple disaster status categories in disaster scene images, and the category probability corresponding to each disaster status category; From the disaster knowledge graph, identify multiple disaster entities corresponding to the multiple disaster status categories, and determine the similarity between each disaster entity and each entity in the disaster knowledge graph; Based on the category probability corresponding to each disaster status category and the similarity, the target entity is determined from each entity in the disaster knowledge graph, and the entity information of the target entity is used as the rescue decision result. Also includes: When the disaster scene image includes multiple disaster status categories such as multiple building damage status categories and multiple building material categories, the rescue decision result is determined based on the rescue decision value; The rescue decision value is: Wherein, C(x) i (x) represents the disaster scene image. i The rescue decision value, U is the set of entities that need to be set in all entities of the disaster knowledge graph, u is the index of U, and the building damage status category is {1,…,K}. ′ }, the building material category is {K ′ +1,…,K}, where j is the index of the disaster status category, map(j) is the disaster entity corresponding to disaster status category j in the disaster knowledge graph, A(map(j),u) is the similarity between entity u and the disaster entity corresponding to disaster status category j in the disaster knowledge graph, and f(x i () is a disaster scene image x obtained based on feature extractor f. i The global feature description, g1 and g2 represent the classifiers, g1(f(x)) i )) j It is the probability corresponding to the damage status category of the building, g2(f(x) i )) j It represents the probability corresponding to the building material category.

2. The proactive rescue decision-making method based on knowledge reasoning according to claim 1, characterized in that, The multiple disaster status categories include multiple building damage status categories and multiple building material categories.

3. The proactive rescue decision-making method based on knowledge reasoning according to claim 1, characterized in that, The acquisition of multiple disaster state categories in the disaster scene image, and the category probability corresponding to each disaster state category, includes: The disaster scene image is input into the visual perception model to obtain multiple disaster state categories in the disaster scene image output by the visual perception model, as well as the category probability corresponding to each disaster state category. The visual perception model is obtained by training an initial visual perception model based on disaster scene image samples and disaster status category labels corresponding to the disaster scene image samples.

4. The proactive rescue decision-making method based on knowledge reasoning according to claim 3, characterized in that, In the case where the multiple disaster status categories in the disaster scene image include multiple building damage status categories and multiple building material categories, it also includes: The first classifier of the initial visual perception model is trained based on disaster scene image samples and building damage status category labels corresponding to the disaster scene image samples, and the loss value of the first classifier is obtained. The second classifier of the initial visual perception model is trained based on disaster scene image samples and building material category labels corresponding to the disaster scene image samples, and the loss value of the second classifier is obtained. The sum of the loss values ​​of the first classifier and the second classifier is determined, and the visual perception model is determined to be converged when the sum is minimized. The visual perception model is obtained by combining the first classifier and the second classifier.

5. The proactive rescue decision-making method based on knowledge reasoning according to claim 1, characterized in that, The step of determining the similarity between each disaster entity and each entity in the disaster knowledge graph includes: Construct an entity similarity matrix based on the similarity between any two entities in the disaster knowledge graph; Based on the entity similarity matrix, the similarity between each disaster entity and each entity in the disaster knowledge graph is determined.

6. The proactive rescue decision-making method based on knowledge reasoning according to claim 5, characterized in that, Determining the similarity between any two entities in the disaster knowledge graph includes: Based on each entity in the disaster knowledge graph, determine the entity vector of each entity and the vector distance between each entity vector; The similarity between any two entities in the disaster knowledge graph is determined based on the vector distance.

7. A proactive rescue decision-making device based on knowledge reasoning, characterized in that, include: The disaster status category determination module is used to obtain multiple disaster status categories in disaster scene images, as well as the category probability corresponding to each disaster status category; The similarity determination module is used to determine multiple disaster entities corresponding to the multiple disaster status categories from the disaster knowledge graph, and to determine the similarity between each disaster entity and each entity in the disaster knowledge graph. The rescue decision determination module is used to determine the target entity from each entity in the disaster knowledge graph based on the category probability corresponding to each disaster status category and the similarity, and to use the entity information of the target entity as the rescue decision result. Also includes: When the disaster scene image includes multiple disaster status categories such as multiple building damage status categories and multiple building material categories, the rescue decision result is determined based on the rescue decision value; The rescue decision value is: Wherein, C(x) i (x) represents the disaster scene image. i The rescue decision value, U is the set of entities that need to be set in all entities of the disaster knowledge graph, u is the index of U, and the building damage status category is {1,…,K}. ′ }, the building material category is {K ′ +1,…,K}, where j is the index of the disaster status category, map(j) is the disaster entity corresponding to disaster status category j in the disaster knowledge graph, A(map(j),u) is the similarity between entity u and the disaster entity corresponding to disaster status category j in the disaster knowledge graph, and f(x i () is a disaster scene image x obtained based on feature extractor f. i The global feature description, g1 and g2 represent the classifiers, g1(f(x)) i )) j It is the probability corresponding to the damage status category of the building, g2(f(x) i )) j It represents the probability corresponding to the building material category.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the proactive rescue decision-making method based on knowledge reasoning as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the knowledge-based proactive rescue decision-making method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the knowledge-based proactive rescue decision-making method as described in any one of claims 1 to 6.

Images