Security information intelligent search method and device, terminal equipment and storage medium

Abstract

The invention relates to the technical field of engineering monitoring, and discloses a security information intelligent search method and device, terminal equipment and a storage medium, the method comprises the following steps: after input retrieval information is received, performing multi-modal fusion retrieval on the retrieval information to obtain a multi-modal retrieval result; reasoning the retrieval information through a hidden danger conduction knowledge graph to obtain a reasoning result; identifying a risk entity in the retrieval information, and obtaining risk information through a risk quantification model; and outputting the multi-modal retrieval result, the reasoning result and the risk information as security information search results. Through three search modes, contents more related to construction site safety are output for the user, so that the output contents better meet the requirements of the user.

Description

Technical Field

[0001] This application relates to the field of engineering monitoring technologies, and in particular, to an intelligent search method, device, terminal device, and storage medium for safety information. Background Art

[0002] With the development of technology, construction site safety has received increasing attention, and some construction site management systems have emerged. Among them, the search function of the current traditional construction site management system has the following defects: relying on keyword matching, unable to understand complex query semantics, lacking the ability to integrate real-time Internet of Things data, and the search results being disconnected from the safety control requirements, making it difficult to return comprehensive search results that meet user needs, and the search results being rigid. Summary of the Invention

[0003] In view of this, embodiments of this application provide an intelligent search method, device, terminal device, and storage medium for safety information, which can effectively solve problems such as rigid search results.

[0004] In a first aspect, embodiments of this application provide an intelligent search method for safety information, including:

[0005] After receiving the input retrieval information, perform multimodal fusion retrieval on the retrieval information to obtain a multimodal retrieval result;

[0006] Through a hidden danger conduction knowledge graph, perform reasoning on the retrieval information to obtain a reasoning result;

[0007] Identify risk entities in the retrieval information, and through a risk quantification model, obtain risk information;

[0008] Output the multimodal retrieval result, the reasoning result, and the risk information as the safety information search result.

[0009] In some embodiments, performing multimodal fusion retrieval on the retrieval information to obtain a multimodal retrieval result includes:

[0010] Perform multimodal feature encoding on the retrieval information and perform enhancement processing to obtain enhanced information;

[0011] According to the enhanced information, perform cross-modal retrieval to obtain a text retrieval result and an image retrieval result.

[0012] In some embodiments, performing multimodal feature encoding on the retrieval information and performing enhancement processing to obtain enhanced information includes:

[0013] If the retrieval information is text information, perform term strengthening and context enhancement on the text information;

[0014] If the retrieved information is image information, identify the key area of the image information and perform multi-scale feature extraction to obtain enhanced features.

[0015] In some embodiments, inferring the retrieved information through the hidden danger conduction knowledge graph to obtain an inference result includes:

[0016] Determine the core entity in the retrieved information from the hidden danger conduction knowledge graph;

[0017] Obtain the relevant data of the core entity and determine the state of the core entity;

[0018] According to the state of the core entity, determine the current risk state of the core entity and generate an inference result related to the risk state.

[0019] In some embodiments, obtaining the relevant data of the core entity and determining the state of the core entity includes:

[0020] Obtain the monitoring data and location data of the core entity, and perform spatio-temporal alignment on the location data;

[0021] Infer the core entity based on the monitoring data and the location data to obtain the physical conduction information, spatial conduction information, and personnel impact information of the core entity.

[0022] In some embodiments, identifying the risk entity in the retrieved information and obtaining risk information through a risk quantification model includes:

[0023] Identify the semantics of the retrieved information and obtain the real-time sensing data of the risk entity in the semantics;

[0024] Calculate the risk quantification characteristics of the risk entity according to the sensing data;

[0025] Construct a spatial superposition model and combine the risk quantification characteristics to simulate and calculate to obtain risk information.

[0026] In some embodiments, calculating the risk quantification characteristics of the risk entity according to the sensing data includes:

[0027] Calculate the physical risk and spatial risk of the risk entity through a preset physical risk model and spatial risk model, and combine time to calculate the time decay of the physical risk and the spatial risk to obtain the risk quantification characteristics of the risk entity.

[0028] In a second aspect, the present application also provides a security information intelligent search device, including:

[0029] A multimodal fusion retrieval module, configured to perform multimodal fusion retrieval on the input retrieval information when receiving the same, so as to obtain a multimodal retrieval result;

[0030] An inference module, configured to perform inference on the retrieval information through a hidden danger conduction knowledge graph to obtain an inference result;

[0031] A risk quantification module, configured to identify risk entities in the retrieval information and obtain risk information through a risk quantification model;

[0032] An output module, configured to output the multimodal retrieval result, the inference result, and the risk information as a security information search result.

[0033] In a third aspect, the present application further provides a terminal device, which includes a processor and a memory. The memory stores a computer program, and the processor is configured to execute the computer program to implement the security information intelligent search method.

[0034] In a fourth aspect, the present application further provides a readable storage medium, which stores a computer program. When the computer program is executed on a processor, the security information intelligent search method is implemented.

[0035] The embodiments of the present application have the following beneficial effects:

[0036] Through three retrieval methods, the present application feeds back a multimodal retrieval result, an inference result, and risk information, so that when a user conducts a search, in addition to the traditional retrieval result, an inference result related to the retrieval content and risk information related to the retrieval content will also be output, enabling the user to obtain more content related to construction site safety in one retrieval, achieving a better retrieval effect, and being able to provide multi-dimensional retrieval data for the user to better master the construction site situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1 Shows a schematic flowchart of a security information intelligent search method according to an embodiment of the present application;

[0039] Figure 2 Shows a risk heat map according to an embodiment of the present application;

[0040] Figure 3The structural schematic diagram of the intelligent security information search device according to the embodiment of the present application is shown. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0042] Generally, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application claimed, but merely represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0043] In the following text, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are only intended to represent specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be construed as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0044] Unless otherwise defined, all terms (including technical terms and scientific terms) used here have the same meaning as commonly understood by those of ordinary skill in the art to which the various embodiments of the present application belong. The terms (such as those defined in a general-use dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0045] The present application provides an intelligent security information search method. Aiming at the problems in the existing search methods such as the lack of real-time Internet of Things data integration ability and the disconnection between search results and security control requirements, when performing a search task, this solution will perform multimodal fusion retrieval, reason about the retrieved information through a hidden danger conduction knowledge graph, and identify it through a risk quantification model to obtain various search results, so that users can receive more comprehensive search results.

[0046] Next, some embodiments of the present application will be described in detail with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0047] Figure 1 FIG. 2 shows a flowchart of a method for intelligent search of security information according to an embodiment of the present application. Exemplarily, the method for intelligent search of security information includes the following steps:

[0048] Step S100, when the input retrieval information is received, perform multimodal fusion retrieval on the retrieval information to obtain a multimodal retrieval result.

[0049] The retrieval information can be common text information or picture information. The text information can be simple keywords, such as "guardrail", "tower crane", etc. Since the method of this embodiment is mainly applied to the scenario of construction site monitoring, for the user's search, search results related to the current construction site will be fed back.

[0050] Among them, this embodiment supports multimodal fusion retrieval, that is, retrieval can be performed on both text and images, and image results and text results can be fed back. For example, when the user inputs the two words "guardrail", the guardrail image of the current construction site and the text retrieval results related to the guardrail can be fed back.

[0051] Specifically, when obtaining the retrieval information, multimodal feature encoding will be performed and enhancement processing will be carried out to obtain enhanced information. Then, according to the enhanced information, cross-modal retrieval will be performed to obtain text retrieval results and image retrieval results.

[0052] For the input retrieval information that is text, corresponding text stream processing will be performed. The words input by the user are not necessarily professional construction terms. Therefore, through means such as association, the words input by the user will be associated with professional terms to enhance the text information. A professional thesaurus can be established, and a corresponding relationship such as "guardrail → protective railing" can be established in the thesaurus. For the text input by the user, multiple keywords can be found, and these keywords are respectively strengthened. The text input by the user can be strengthened into a standardized construction term for subsequent retrieval and analysis. In addition to strengthening the words, context enhancement will also be performed. For example, if the user may input a long sentence, the context will be combined for comprehensive analysis to further understand the user's retrieval intention.

[0053] For the input retrieval information that is an image, key area focusing processing will be performed, such as locating the part of the image related to construction site equipment, and multi-scale feature extraction will be performed, such as capturing the global features of the overall structure through ResNet, and detecting the local features in the image through Mask R-CNN. In addition, for some features related to construction site safety, enhancement processing will be performed, such as performing gradient backpropagation enhancement on some defects to strengthen these features.

[0054] After the above processing, enhanced text information or image information is obtained. Such enhanced information can better highlight the key features in the retrieval information, facilitating subsequent retrieval work.

[0055] For the enhanced information, cross-modal retrieval is performed to obtain text retrieval results and image retrieval results.

[0056] Regarding cross-modal retrieval, it means achieving the retrieval of associated images through text and the reverse retrieval of associated text information through images. It should be noted that the images searched through text can be construction site-related images stored in the server through monitoring, personnel upload, or other means. By mapping the query text to the visual space and calculating the Top-K similar images, relevant images in the text can be queried. For example, when a user queries "guardrail condition", in addition to feedbacking relevant text information such as the maintenance log of the guardrail, matching guardrail images will also be feedbacked. If the retrieval information contains a specific location and a corresponding image is matched, the guardrail image at the specific location can be feedbacked.

[0057] If the input is image information, after extracting the features, the associated log records can be retrieved in reverse through the features, and then these log records are feedbacked, thus achieving the effect of image-to-text search.

[0058] In addition, to ensure the validity of the feedbacked data, only the data within a recent period of time can be feedbacked. For example, only the data within the most recent 3 weeks can be feedbacked, or the data 3 weeks ago can be down-weighted to reduce the feedback of old data and try to feedback the latest data as much as possible.

[0059] It can be seen that through the multi-modal retrieval of this embodiment, the retrieval information input by the user can be flexibly processed, and relevant data that the user cares about can be output as much as possible. And the feedbacked retrieval data is not limited to text, enabling the user to more intuitively obtain the information they want from the retrieval results.

[0060] Step S200, through the hidden danger conduction knowledge graph, reason about the retrieval information to obtain a reasoning result.

[0061] In a construction site, there are often various hidden dangers and risks. The application scenario of the search method in this embodiment is often to deal with these risks. When a user conducts a search, in addition to understanding the situation of the construction site, the most important thing is to know where there are hidden dangers on the construction site. Therefore, when the user conducts a retrieval, hidden danger-related processing can be carried out to give the user hidden danger-related reference opinions.

[0062] For example, when a user conducts a search, they may not only search for "XX facility", but may also search for statements such as "How about the safety hidden danger of XX facility". Therefore, the search method is required to be able to answer such questions in combination with the current construction site status.

[0063] The hidden danger conduction knowledge graph is a relational graph formed by associating core entities and related relationships. The core entities include:

[0064] Physical entities, such as tower cranes (model / inclination angle / status of torque limiter).

[0065] Spatial entities, such as working radius / restricted area / temporary passage.

[0066] Personnel entities, such as job types (signalman / slinger) / positioning coordinates.

[0067] Event entities, such as illegal operations / extreme weather warnings.

[0068] These entities will all have their relevant association relationships. For example, a tower crane has a working space, so it is associated with a spatial area. A tower crane also has a real-time status, and its current inclination angle, etc., can represent its real-time status. For various entities on the construction site, their respective hidden danger conduction knowledge graphs will be established. Through the hidden danger conduction knowledge graph, appropriate reasoning can be carried out to determine the risks of each core entity.

[0069] When the user inputs retrieval information, through text processing, it can be determined which relevant core entities are in the text. Then, when reasoning, it is possible to only reason about the core entities that appear in the text to give relevant reasoning results.

[0070] As mentioned above, to determine the core entities, relevant data in the associated ge can be determined through the hidden danger conduction knowledge graph. These data can be obtained through sensors on the construction site or log data recorded in the database, so as to determine the status of the core entities.

[0071] For example, if the user inputs "tower crane", through the sensors of the tower crane itself, monitoring data can be obtained. For example, the inclination angle of the tower crane can be obtained through an inclination sensor, and the current load can be obtained through a load sensor. And through log data or positioning devices set on the construction site, the position data of the tower crane can be obtained. The detection data can reflect the current status of the tower crane, and the position data can determine the influence range of the tower crane. Then, based on these two data, appropriate reasoning can be carried out to obtain physical conduction information, spatial conduction information, and personnel influence information.

[0072] The physical conduction information can be used to real-time calculate the hook trajectory envelope through tower crane kinematic modeling, so that the collision area can be predicted.

[0073] The spatial conduction information can be obtained by dividing the construction site into multiple grids and assigning a risk value to each grid to give the risk areas that the tower crane may affect on the construction site.

[0074] The personnel impact information refers to the risk levels faced by personnel at different locations. For example, through a positioning program, the user who inputs the retrieval information can be obtained, and the user can be informed whether the current location is at risk of being affected by tower cranes.

[0075] It can be understood that through the above physical conduction information, spatial conduction information, and personnel impact information, it is possible to more specifically determine whether there are safety hazards exceeding the regulations for the core entity, and users can also intuitively understand the impact of the core entity they retrieved in the construction site, thereby intuitively grasping the safety status of the construction site.

[0076] To better reflect the risk impact of the core entity in the construction site, a danger heat map can be used for tactics. As shown before, the spatial conduction information divides the construction site into grids, and each grid is assigned a risk value. Then, the risk value can be gradually changed in color from high to low. For example, high risk is red, and low risk is white or green, etc., to reflect the affected area.

[0077] Exemplarily, as Figure 2 shown, it is a possible form of the danger heat map. In the figure, a construction site map is established in the form of grids, and the locations of tower cranes are marked. Among them, the areas with darker colors around the tower cranes are considered areas with higher danger levels, the areas with lighter colors are areas with medium danger levels, and the white areas are risk-free areas. By providing such a map, users can know the risk impact range of tower cranes in the current construction site.

[0078] Among them, the above-mentioned hidden danger conduction knowledge graph can be updated in real time through manual or online updates by personnel to ensure the accuracy of each relationship in the hidden danger conduction knowledge graph.

[0079] Step S300, identify the risk entities in the retrieval information, and obtain risk information through a risk quantification model.

[0080] This embodiment can also give quantified risk information through a risk quantification model.

[0081] When the user inputs retrieval information, risk entities can be obtained from the retrieval information through semantic recognition. These risk entities are similar to the aforementioned core entities and are entities that will cause specific risks in the construction site. It can be understood that if there are no risk entities in the retrieval information input by the user, the processing process of the risk quantification model in this step will not be triggered.

[0082] After determining the risk entities, relevant data needs to be obtained. Here, various real-time sensing data can be obtained through the sensors of the risk entities themselves. These sensing data will be sent to the server, so as long as it is connected to the relevant server, these data can be obtained.

[0083] Then, based on these data, the risk quantification features can be calculated.

[0084] Still taking the "tower crane" as an example, the risk quantification features include physical risk and spatial risk. Through the preset physical risk model and spatial risk model, these two types of risks can be calculated respectively.

[0085] For a tower crane, its physical risks can be inclination risk and load risk, and its spatial risk can be collision risk. In this embodiment, this risk can be quantitatively calculated through prior knowledge.

[0086] For example, for the inclination risk, its calculation expression can be: R_θ = e ( Measured inclination angle / Safety threshold )-1 .

[0087] In the formula, R_θ is the inclination risk and e is the natural logarithm.

[0088] For the load risk, its calculation expression can be: R_w = Real-time load / Rated load × Dynamic load factor.

[0089] In the formula, R_w is the load risk.

[0090] For the collision risk, its calculation expression can be: R_c = ∑(Obstacle mass × Relative speed²) / Safety distance.

[0091] In the formula, R_c is the collision risk.

[0092] The various coefficients and formula structures in the above expressions can be established in a prior manner, and then the risk quantification models of various risk entities can be constructed based on this. For different risk entities, their quantification models will vary, but the physical risk or spatial risk of each can be quantitatively calculated. Here, only the risk quantification model of the tower crane is taken as an example for display, and there will be corresponding calculation formulas for other risk entities, which will not be exemplified one by one here.

[0093] In addition, the risk will decay over time. Therefore, for each risk, the real-time risk can be calculated through a time decay function.

[0094] The expression of the time decay function is:

[0095] R(t) = R_0 × [1 - 1 / (1 + e (-k(t-t0)) ;

[0096] In the formula, R(t) is the risk value at the current moment t, R_0 is the risk value at the moment t0, and k is a hidden danger type parameter, which varies with the hidden danger type.

[0097] Among them, the above R_0 can be any one of the aforementioned physical risks or spatial risks. In this way, the quantification features can be obtained.

[0098] In addition, in order to represent the degree of danger of this risk relative to personnel, a spatial superposition model can be established. For example, a 3D voxel model representing a person is established, and its risk degree is calculated based on the physical risk, environmental risk, and personnel exposure degree at the location where the 3D voxel model is located, so as to realize the quantification of the risk for people. Among them, the physical risk refers to the risk brought by the aforementioned risk entity, the environmental risk is the basic risk brought by the environment where the 3D voxel model is located, and the personnel exposure degree is determined according to whether the personnel wear safety helmets and whether they are indoors, etc.

[0099] Through the above method, after receiving the retrieval information input by the user, the corresponding risk calculation can be performed on the risk entity therein, and a quantified risk calculation result can be obtained. This quantified risk information is different from the inference information given by the inference in S200. The risk information has more mathematical significance and can be compared with historical data, enabling the user to intuitively feel the increase or decrease of the risk, and being able to give corresponding risk prompts more objectively and in line with the actual situation.

[0100] Step S400, output the multi-modal retrieval result, the inference result, and the risk information as the safety information search result.

[0101] The above multi-modal retrieval result, inference result, and risk information will all be output as the safety information search result. It can be understood that the intelligent search method of this embodiment can run on an application program, or on a web page or a small program. After the search, a search result page can be specifically popped up, and all the search results are presented by loading the search results on this search result page.

[0102] The intelligent search method for safety information of this embodiment retrieves, infers, and calculates in three ways, can perform corresponding processing based on the pictures or texts input by the user, combines the actual situation of the construction site, and outputs feedback results related to construction site safety, enabling the user to obtain a large amount of effective search result data through simple input, and these data are closely related to the current construction site, ensuring the real-time effectiveness of the data, being closely related to the safety control requirements, and having the ability to integrate real-time Internet of Things data, and can give corresponding search results according to the status in the current construction site.

[0103] Figure 3 FIG. shows a schematic structural diagram of an intelligent search device for safety information according to an embodiment of the present application. Exemplarily, the intelligent search device for safety information includes:

[0104] A multi-modal fusion retrieval module 10, configured to perform multi-modal fusion retrieval on the retrieval information when receiving the input retrieval information, so as to obtain a multi-modal retrieval result;

[0105] An inference module 20, configured to perform inference on the retrieved information through a hidden danger conduction knowledge graph to obtain an inference result;

[0106] A risk quantification module 30, configured to identify risk entities in the retrieved information and obtain risk information through a risk quantification model;

[0107] An output module 40, configured to output the multimodal retrieval result, the inference result, and the risk information as a security information search result.

[0108] It can be understood that the device in this embodiment corresponds to the security information intelligent search method in the above embodiment. The optional items in the above embodiment are also applicable to this embodiment, so they will not be described again here.

[0109] The present application further provides a terminal device. Exemplarily, the terminal device includes a processor and a memory. The memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of the above security information intelligent search method or each module in the above security information intelligent search device.

[0110] Among them, the processor may be an integrated circuit chip with signal processing capabilities. The processor may be a general-purpose processor, including at least one of a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc., and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application.

[0111] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (EEPROM), etc. Among them, the memory is used to store a computer program, and after receiving an execution instruction, the processor can execute the computer program accordingly.

[0112] This application also provides a readable storage medium for storing the computer program used in the above terminal device.

[0113] In several embodiments provided by this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and structure diagrams in the drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the blocks can occur in a different order than marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the structure diagram and / or flowchart, as well as the combination of blocks in the structure diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0114] In addition, each functional module or unit in various embodiments of this application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0115] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0116] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application.

Claims

1. A security information intelligent search method, characterized in that: include: After receiving the input search information, performing a multimodal fusion search on the search information to obtain a multimodal search result; Reasoning the search information through the hidden danger transmission knowledge graph to obtain a reasoning result; Identify risk entities in the retrieved information, and obtain risk information through a risk quantification model; The multimodal retrieval result, the reasoning result and the risk information are output as security information search results.

2. The security information intelligent search method according to claim 1, characterized in that: The search information is subjected to multimodal fusion search to obtain multimodal search results, including: Performing multimodal feature encoding on the search information and performing enhancement processing to obtain enhanced information; Based on the enhanced information, a cross-modal retrieval is performed to obtain text retrieval results and image retrieval results.

3. The security information intelligent search method according to claim 2 is characterized in that: The performing multimodal feature encoding on the search information and performing enhancement processing to obtain enhanced information includes: If the search information is text information, term enhancement and context enhancement are performed on the text information; If the retrieval information is image information, the key area of ​​the image information is identified and multi-scale feature extraction is performed to obtain enhanced features.

4. The security information intelligent search method according to claim 1, characterized in that: The hidden danger transmission knowledge graph is used to infer the search information to obtain the inference results, including: Determine the core entity in the search information from the hidden danger transmission knowledge graph; Acquire relevant data of the core entity and determine the status of the core entity; According to the state of the core entity, the current risk state of the core entity is determined, and a reasoning result related to the risk state is generated.

5. The security information intelligent search method according to claim 4 is characterized in that: The acquiring relevant data of the core entity and determining the state of the core entity includes: Acquire monitoring data and location data of the core entity, and perform spatiotemporal alignment on the location data; The core entity is inferred based on the monitoring data and the location data to obtain physical conduction information, spatial conduction information and personnel impact information of the core entity.

6. The security information intelligent search method according to claim 1, characterized in that: The step of identifying the risk entity in the retrieved information and obtaining risk information through a risk quantification model includes: Identify the semantics of the search information and obtain real-time sensor data of risk entities in the semantics; Calculating the risk quantification characteristics of the risk entity according to the sensor data; A spatial superposition model is constructed and combined with the risk quantification characteristics, risk information is obtained through simulation and calculation.

7. The security information intelligent search method according to claim 6, characterized in that: The step of calculating the risk quantification feature of the risk entity according to the sensor data includes: The physical risk and spatial risk of the risk entity are calculated by using a preset physical risk model and a spatial risk model, and the time decay of the physical risk and the spatial risk is calculated in combination with time to obtain the risk quantification characteristics of the risk entity.

8. A security information intelligent search device, characterized in that: include: A multimodal fusion retrieval module is used to perform a multimodal fusion retrieval on the retrieval information after receiving the input retrieval information, so as to obtain a multimodal retrieval result; A reasoning module, used to reason about the search information through the hidden danger transmission knowledge graph to obtain a reasoning result; A risk quantification module, used to identify risk entities in the search information and obtain risk information through a risk quantification model; An output module is used to output the multimodal retrieval result, the reasoning result and the risk information as security information search results.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the computer program to implement the security information intelligent search method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: It stores a computer program, which, when executed on a processor, implements the security information intelligent search method according to any one of claims 1-7.

Images