Railway geological disaster early warning method, device, equipment, medium and product

Abstract

The invention discloses a railway geological disaster early warning method, device and equipment, a medium and a product. The invention discloses a railway geological disaster early warning method. The method comprises the following steps: acquiring a railway scene picture and a cue word text pair; according to the railway scene picture and the cue word text pair, training a multi-modal large model to obtain a railway geological disaster recognition model; and according to the railway geological disaster identification model and the real-time railway scene picture, identifying the railway geological disaster, and when the existence of the railway geological disaster is identified, carrying out early warning. According to the technical scheme provided by the embodiment of the invention, a disaster detection problem is ingeniously converted into a scene understanding problem with unlimited categories, and accurate early warning of railway geological disasters is well realized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a railway geological disaster early warning method, device, equipment, medium and product. Background Art

[0002] High-speed railways are equipped with natural disaster and foreign object intrusion monitoring systems to conduct real-time detection of wind, rain, snow, earthquake and foreign object intrusion events along the railway, and have formulated corresponding design, construction and maintenance specifications. However, in addition to being affected by wind, rain, snow, foreign object intrusion and earthquake disasters, railways may also be subject to safety threats such as mudslides, floods or sudden invasion of dangerous sources at any time. In particular, geological disasters are prone to occur near tunnel entrances, such as floods, landslides, tunnel entrance collapse, mudslides, etc., which seriously affect people's lives and property safety. Therefore, safety monitoring of tunnel entrances and timely early warning and alarm can effectively prevent accidents and ensure the efficiency and safety of railway transportation.

[0003] At present, the main technical means for monitoring mudslides and flood disaster scenes at tunnel entrances include GNSS (Global Navigation Satellite System), lidar, optical fences and video analysis.

[0004] The design concept of GNSS is to use artificial satellites in space as reference points to determine the spatial position of an object. According to geometric theory, the distance between a certain point on the earth and three artificial satellites is accurately measured to determine the position of this point. The phase difference between the monitoring station and the base station is used to calculate whether displacement occurs. This technical equipment is only suitable for single-point monitoring scenarios.

[0005] The working principle of LiDAR is to use visible light and near-infrared light to emit a signal, which is reflected by the target and collected by the receiving system. The distance of the target is determined by measuring the running time of the emitted light. The radial velocity of the target can be determined by the Doppler shift of the reflected light. This technology has high precision and accuracy, but the monitoring range of a single LiDAR is small.

[0006] Fiber optic fence monitoring technology uses the "photoelastic effect" mechanism to detect the phase change of light waves caused by vibration and identify the cause of the vibration through the change. This technology has the characteristics of high sensitivity and rapid response, but it is easily affected by natural environmental interference factors and has high deployment costs.

[0007] The basic process of video analysis target detection is to read each frame of the given video, pre-process the input image, such as filtering, grayscale conversion, etc., then determine whether there is a moving target in the input image, determine whether the moving target is a monitoring target, and finally monitor, track or analyze the target. This technology uses AI (Artificial Intelligence) analysis with a high accuracy rate, but requires a large number of data samples to train the detection model to ensure its detection accuracy.

[0008] However, GNSS is easily affected by severe weather such as wind, rain and snow, as well as obstructions from clouds and trees and vegetation; video analysis methods based on target detection require a large amount of scene data, while railway disaster data is extremely difficult to obtain; radar and fiber optic fences are not affected by severe environments such as rain, snow and fog, but the alarms are not intuitive and it is difficult to determine the type of monitored object. Summary of the invention

[0009] The present invention provides a railway geological disaster early warning method, device, equipment, medium and product to solve the problem of difficulty in obtaining geological disaster data in railway scenarios, which leads to poor railway geological disaster detection and classification performance.

[0010] According to one aspect of the present invention, there is provided a railway geological disaster early warning method, comprising:

[0011] Get the railway scene picture and prompt word text pair;

[0012] Based on the railway scene pictures and prompt word text pairs, a multimodal large model is trained to obtain a railway geological disaster recognition model;

[0013] Railway geological disasters are identified based on the railway geological disaster identification model and real-time railway scene pictures, and early warnings are issued when railway geological disasters are identified.

[0014] According to another aspect of the present invention, there is provided a railway geological disaster early warning device, comprising:

[0015] A data acquisition module, used to acquire railway scene images and prompt word text pairs;

[0016] The model training module is used to train a multimodal large model based on railway scene pictures and prompt word text pairs to obtain a railway geological disaster recognition model;

[0017] The disaster warning module is used to identify railway geological disasters based on the railway geological disaster identification model and real-time railway scene pictures, and to issue warnings when railway geological disasters are identified.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the railway geological disaster early warning method described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the railway geological disaster early warning method described in any embodiment of the present invention when executed.

[0023] According to another aspect of the present invention, a computer program product is provided, characterized in that the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the railway geological disaster early warning method described in any embodiment of the present invention.

[0024] The technical solution of the embodiment of the present invention obtains a railway scene image and a prompt word text pair, and then trains a multimodal large model according to the railway scene image and the prompt word text pair to obtain a railway geological disaster recognition model, and then identifies the railway geological disaster according to the railway geological disaster recognition model and the real-time railway scene image, and issues an early warning when the railway geological disaster is identified. In this solution, the advantages of the multimodal large language model are utilized to convert the disaster detection problem into a scene understanding problem with no restrictions on categories, so as to learn the alignment relationship between images and texts through the multimodal large language model, realize cross-modal information interaction and fusion, and ensure that the railway geological disaster can be accurately identified and warned without learning new pictures, which greatly reduces the demand for sample pictures in the railway geological disaster scene, solves the problem of poor railway geological disaster detection and classification performance caused by the difficulty in obtaining geological disaster data in the railway scene, and can cleverly convert the disaster detection problem into a scene understanding problem with no restrictions on categories, and well realizes accurate early warning of railway geological disasters.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A flow chart of a railway geological disaster early warning method provided in Embodiment 1 of the present invention;

[0028] Figure 2 A flow chart of a railway geological disaster early warning method provided in Embodiment 2 of the present invention;

[0029] Figure 3 A schematic diagram of the structure of a railway geological disaster early warning device provided in Embodiment 3 of the present invention;

[0030] Figure 4 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0033] Embodiment 1

[0034] Figure 1 This is a flow chart of a railway geological disaster early warning method provided in the first embodiment of the present invention. This embodiment can be applied to accurately warn railway geological disasters in the absence of sample data. The method can be executed by a railway geological disaster early warning device, which can be implemented in the form of hardware and / or software. The railway geological disaster early warning device can be configured in an electronic device. The form of the electronic device includes but is not limited to a computer or a server. Figure 1As shown, the method includes:

[0035] Step 110: Obtain a railway scene image and a prompt word text pair.

[0036] The railway scene image and the prompt word text pair may be data matched in pairs by the railway scene image and the prompt word text. Optionally, in order to be able to effectively and targetedly identify railway geological disasters, the railway scene images include not only railway images in a normal train running environment, but also railway images suffering from geological disasters. The prompt word text can be understood as a text description of the railway scene that matches the railway scene image.

[0037] In the embodiment of the present invention, a prompt word text that meets the need for railway geological disaster identification and a railway scene image object corresponding to the prompt word text can be obtained to obtain a railway scene image and prompt word text pair.

[0038] Step 120: Train a multimodal large model based on the railway scene images and prompt word text pairs to obtain a railway geological disaster recognition model.

[0039] Among them, the multimodal large model can be a model obtained by expanding the large language model to multiple data types, that is, a fusion model of multimodality and large models. The multimodal large model can include but is not limited to CLIP (Contrastive Language-Image Pre-training, text and image pre-training) model, CoCa (Contrastive Captioner, multimodal image-text basic model) and CoDi (combinable diffusion model). The railway geological disaster identification model can be a multimodal large model trained using railway scene pictures and prompt word text pairs.

[0040] In an embodiment of the present invention, a railway scene image and a prompt word text pair can be used to train a multimodal large model, and the trained multimodal large model can be used as a railway geological disaster recognition model.

[0041] Step 130: Identify railway geological disasters based on the railway geological disaster identification model and the real-time railway scene image, and issue an early warning when railway geological disasters are identified.

[0042] Among them, the real-time railway scene picture can be a railway scene picture collected in real time by cameras deployed on the railway running line.

[0043] In an embodiment of the present invention, real-time railway scene images can be input into a railway geological disaster identification model to analyze the real-time railway scene images based on the railway geological disaster identification model to identify railway geological disasters. When the identification result output by the railway geological disaster identification model is that a geological disaster exists, an early warning is issued.

[0044] For example, the CLIP model can be trained based on railway scene images and prompt word text pairs to obtain a railway geological disaster recognition model. The CLIP model consists of two main parts: an image encoder and a text encoder. The image encoder is responsible for converting images into feature vectors, usually based on a convolutional neural network or a Transformer model; the text encoder is responsible for converting text into feature vectors, usually a Transformer model. The two encoders achieve cross-modal information interaction and fusion by sharing a vector space. In the pre-training stage of the CLIP model, through contrastive learning training, matching image and text vectors are brought closer in a common semantic space, while mismatched ones are pushed apart.

[0045] By training with large-scale railway scene images and prompt word text pairs, the CLIP model can learn general visual semantic information to provide guidance in downstream tasks, especially in zero-shot learning tasks, where the CLIP model can make predictions without seeing new image or text training examples. It has been verified that the CLIP model has good performance in zero-shot learning tasks and can achieve early warning of disaster scenes such as mudslides and floods at the entrance of railway tunnels. Unlike the fixed number of categories in previous classification networks, the CLIP model has a high degree of freedom to set the classification labels provided to the network by "multiple-choice questions". Not only is the number not fixed, but the content is also free, thus getting rid of the constraints of traditional classification models that need to define the categories to be classified. According to the disaster categories of actual scenes, sort out and analyze.

[0046] The technical solution of the embodiment of the present invention obtains a railway scene image and a prompt word text pair, and then trains a multimodal large model according to the railway scene image and the prompt word text pair to obtain a railway geological disaster recognition model, and then identifies the railway geological disaster according to the railway geological disaster recognition model and the real-time railway scene image, and issues an early warning when the railway geological disaster is identified. In this solution, the advantages of the multimodal large language model are utilized to convert the disaster detection problem into a scene understanding problem with no restrictions on categories, so as to learn the alignment relationship between images and texts through the multimodal large language model, realize cross-modal information interaction and fusion, ensure that the railway geological disaster can be accurately identified and warned without learning new pictures, greatly reduce the demand for sample pictures in the railway geological disaster scene, solve the problem of poor railway geological disaster detection and classification performance caused by the difficulty in obtaining geological disaster data in the railway scene, and cleverly convert the disaster detection problem into a scene understanding problem with no restrictions on categories, and well realize accurate early warning of railway geological disasters.

[0047] Embodiment 2

[0048] Figure 2This is a flow chart of a railway geological disaster early warning method provided in the second embodiment of the present invention. This embodiment is specific based on the above embodiment and provides a specific optional implementation method for early warning when the railway geological disaster is identified. Figure 2 As shown, the method includes:

[0049] Step 210: Obtain a railway scene image and a prompt word text pair.

[0050] In an optional embodiment of the present invention, the prompt word text in the railway scene picture and prompt word text pair may include: railway pictures without geological disasters, railway pictures without geological disasters including trains, railway pictures including flood disasters, railway disaster pictures including mudslide disasters, flood disaster pictures and mudslide disaster pictures.

[0051] In an embodiment of the present invention, image objects corresponding to the prompt word text (e.g., railway images without geological disasters, railway images without geological disasters including trains, railway images including flood disasters, railway disaster images including debris flow disasters, flood disaster images, and debris flow disaster images) can be collected from the Internet or images taken by cameras along the railway (railway image objects when there are no geological disasters, railway image objects when there are flood disasters, railway image objects when there are debris flow disasters, image objects of flood disaster scenes, and image objects of debris flow disaster scenes). When the railway scene image and the prompt word text of the prompt word text are the above content, accurate early warning of debris flow and flood disasters can be achieved.

[0052] Step 220: Train a multimodal large model based on the railway scene image and prompt word text pairs to obtain a railway geological disaster recognition model.

[0053] In an optional embodiment of the present invention, after training a multimodal large model based on railway scene images and prompt word text pairs to obtain a railway geological disaster identification model, it can also include: extracting the predicted category text of the railway geological disaster identification model; obtaining the model output category requirement data, and fine-tuning the railway geological disaster identification model based on the predicted category text and the model output category requirement data.

[0054] The predicted category text may be the prompt word text in the pair of the railway scene image and the prompt word text. The model output category requirement data may be data set by the user based on the recognition needs, and control the final output text type of the railway geological disaster recognition model.

[0055] In an embodiment of the present invention, the prediction category text of the railway geological disaster identification model can be extracted, and then the model output category requirement data set by the user can be obtained, and the model output category requirement data can be parsed to determine whether the prediction category text can meet the user's identification needs. If it cannot be met, the railway geological disaster identification model is fine-tuned according to the model output category requirement data.

[0056] In an optional embodiment of the present invention, fine-tuning the railway geological disaster identification model according to the prediction category text and the model output category requirement data may also include: determining the prediction category requirement text according to the model output category requirement data; and when the prediction category requirement text does not match the prediction category text, fine-tuning the railway geological disaster identification model based on the prediction category requirement text.

[0057] The predicted category requirement text may be a prompt word text that is determined based on the model output category requirement data and meets the user's needs.

[0058] In an embodiment of the present invention, the model output category requirement data can be parsed to obtain the prediction category requirement text, so that when the prediction category requirement text is different from the prediction category text, it is determined that the prediction category requirement text and the prediction category text do not match. Based on the prediction category requirement text, the model prediction text type of the railway geological disaster identification model is adjusted to achieve fine-tuning of the railway geological disaster identification model.

[0059] In an optional embodiment of the present invention, the prediction category demand text may include railway pictures without geological disasters, railway pictures without geological disasters including trains, railway pictures with geological disasters, and pictures of geological disasters.

[0060] In an embodiment of the present invention, when the prediction category requirement text is specifically a railway picture without geological disasters, a railway picture without geological disasters including a train, a railway picture with geological disasters, and a railway picture with geological disasters, according to the prediction category requirement text, the railway geological disaster recognition model can cluster and recognize the railway scene pictures in the railway scene picture and the prompt word text pair, that is, cluster and recognize the railway disaster picture objects including debris flow disasters, as well as the flood disaster picture and the debris flow disaster picture objects, and map the output results of the above two types of pictures with the text of the railway picture with geological disasters. Cluster and recognize the flood disaster picture objects and the debris flow disaster picture objects, and map the output results of the two types of pictures with the text of the railway picture with geological disasters. Cluster and recognize the flood disaster picture objects and the debris flow disaster picture objects, and map the output results of the two types of pictures with the text of the geological disaster picture.

[0061] Step 230: Identify railway geological disasters based on the railway geological disaster identification model and the real-time railway scene image, and when the railway geological disaster is identified, determine the target prediction text output by the railway geological disaster identification model.

[0062] Among them, the target prediction text can be the classification text corresponding to the real-time railway scene image object identified by the railway geological disaster identification model.

[0063] In an embodiment of the present invention, after the real-time railway scene image is input into the railway geological disaster recognition model, the railway geological disaster recognition model can be used to identify the railway geological disaster to obtain the target prediction text output by the railway geological disaster recognition model.

[0064] Step 240: Obtain the associated warning data of the real-time railway scene image, and issue a warning for railway geological disasters based on the associated warning data and the target prediction text.

[0065] The associated warning data may be data associated with the real-time railway scene image object. Exemplarily, the associated warning data may include but is not limited to the shooting time of the real-time railway scene image object and the railway location information corresponding to the real-time railway scene image object.

[0066] In an embodiment of the present invention, the associated warning data of the real-time railway scene image can be obtained, and then the associated warning data and the target prediction text can be summarized to generate a warning report, and then the warning report can be reported to achieve real-time intelligent warning of railway geological disasters.

[0067] The technical solution of the embodiment of the present invention obtains a railway scene image and a prompt word text pair, and then trains a multimodal large model according to the railway scene image and the prompt word text pair to obtain a railway geological disaster recognition model, and then identifies the railway geological disaster according to the railway geological disaster recognition model and the real-time railway scene image, and when the railway geological disaster is identified, the target prediction text output by the railway geological disaster recognition model is determined, and the associated warning data of the real-time railway scene image is further obtained, and the railway geological disaster is warned according to the associated warning data and the target prediction text. In this scheme, the advantages of the multimodal large language model are used to convert the disaster detection problem into a scene understanding problem with no restrictions on categories, so as to learn the alignment relationship between images and texts through the multimodal large language model, realize cross-modal information interaction and fusion, ensure that the railway geological disaster can be accurately identified and warned without learning new pictures, greatly reduce the demand for sample pictures in the railway geological disaster scene, solve the problem of poor railway geological disaster detection and classification performance caused by the difficulty in obtaining geological disaster data in the railway scene, and can cleverly convert the disaster detection problem into a scene understanding problem with no restrictions on categories, and well realize accurate warning of railway geological disasters.

[0068] Embodiment 3

[0069] Figure 3This is a schematic diagram of the structure of a railway geological disaster early warning device provided by the third embodiment of the present invention. Figure 3 As shown, the device comprises:

[0070] The data acquisition module 310 is used to acquire a railway scene image and a prompt word text pair;

[0071] The model training module 320 is used to train a multi-modal large model based on the railway scene image and the prompt word text pair to obtain a railway geological disaster recognition model;

[0072] The disaster warning module 330 is used to identify railway geological disasters based on the railway geological disaster identification model and real-time railway scene pictures, and to issue a warning when the existence of railway geological disasters is identified.

[0073] The technical solution of the embodiment of the present invention obtains a railway scene image and a prompt word text pair, and then trains a multimodal large model according to the railway scene image and the prompt word text pair to obtain a railway geological disaster recognition model, and then identifies the railway geological disaster according to the railway geological disaster recognition model and the real-time railway scene image, and issues an early warning when the railway geological disaster is identified. In this solution, the advantages of the multimodal large language model are utilized to convert the disaster detection problem into a scene understanding problem with no restrictions on categories, so as to learn the alignment relationship between images and texts through the multimodal large language model, realize cross-modal information interaction and fusion, and ensure that the railway geological disaster can be accurately identified and warned without learning new pictures, which greatly reduces the demand for sample pictures in the railway geological disaster scene, solves the problem of poor railway geological disaster detection and classification performance caused by the difficulty in obtaining geological disaster data in the railway scene, and can cleverly convert the disaster detection problem into a scene understanding problem with no restrictions on categories, and well realizes accurate early warning of railway geological disasters.

[0074] Optionally, the railway geological disaster early warning device also includes a model fine-tuning module, which is used to extract the prediction category text of the railway geological disaster identification model; obtain the model output category requirement data, and fine-tune the railway geological disaster identification model according to the prediction category text and the model output category requirement data.

[0075] Optionally, the prompt word text in the railway scene picture and prompt word text pair includes: railway pictures without geological disasters, railway pictures without geological disasters including trains, railway pictures including flood disasters, railway disaster pictures including mud-rock flow disasters, flood disaster pictures and mud-rock flow disaster pictures.

[0076] Optionally, a model fine-tuning module is specifically used to determine the prediction category requirement text according to the model output category requirement data; when the prediction category requirement text does not match the prediction category text, fine-tune the railway geological disaster identification model based on the prediction category requirement text.

[0077] Optionally, the prediction category requirement text includes railway pictures without geological disasters, railway pictures without geological disasters including trains, railway pictures with geological disasters, and pictures of geological disasters.

[0078] Optionally, the disaster warning module 330 is specifically used to determine the target prediction text output by the railway geological disaster identification model when the existence of railway geological disasters is identified; obtain the associated warning data of the real-time railway scene image, and issue a warning for the railway geological disaster based on the associated warning data and the target prediction text.

[0079] The railway geological disaster early warning device provided in the embodiment of the present invention can execute the railway geological disaster early warning method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0080] Embodiment 4

[0081] Figure 4 A schematic diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0082] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0083] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0084] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a railway geological disaster early warning method.

[0085] In some embodiments, the railway geological disaster early warning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the railway geological disaster early warning method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the railway geological disaster early warning method by any other appropriate means (for example, by means of firmware).

[0086] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0087] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program product is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0088] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0089] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0090] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0091] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of traditional physical hosts and VPS servers, which are difficult to manage and have weak business scalability.

[0092] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0093] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A railway geological disaster early warning method, characterized in that: include: Get the railway scene picture and prompt word text pair; According to the railway scene picture and prompt word text pair, a multimodal large model is trained to obtain a railway geological disaster recognition model; Railway geological disasters are identified based on the railway geological disaster identification model and real-time railway scene pictures, and an early warning is issued when the existence of railway geological disasters is identified.

2. The method according to claim 1, characterized in that After training a multimodal large model according to the railway scene picture and the prompt word text pair to obtain a railway geological disaster identification model, the method further includes: Extracting the prediction category text of the railway geological disaster identification model; The model output category requirement data is obtained, and the railway geological disaster identification model is fine-tuned according to the prediction category text and the model output category requirement data.

3. The method according to claim 1, characterized in that: The prompt word text in the pair of railway scene pictures and prompt word texts includes: railway pictures without geological disasters, railway pictures without geological disasters including trains, railway pictures including flood disasters, railway disaster pictures including mud-rock flow disasters, flood disaster pictures and mud-rock flow disaster pictures.

4. The method according to claim 2, characterized in that: The fine-tuning of the railway geological disaster identification model according to the prediction category text and the model output category requirement data also includes: Determining predicted category demand text according to the model output category demand data; When the prediction category requirement text does not match the prediction category text, the railway geological disaster identification model is fine-tuned based on the prediction category requirement text.

5. The method according to claim 4, characterized in that The prediction category requirement text includes railway pictures without geological disasters, railway pictures without geological disasters including trains, railway pictures with geological disasters, and pictures of geological disasters.

6. The method according to claim 1, characterized in that When a railway geological disaster is identified, an early warning is issued, including: When the railway geological disaster is identified, determining the target prediction text output by the railway geological disaster identification model; The associated warning data of the real-time railway scene image is obtained, and according to the associated warning data and the target prediction text, a warning is issued for railway geological disasters.

7. A railway geological disaster early warning device, characterized in that: include: A data acquisition module, used to acquire railway scene images and prompt word text pairs; A model training module is used to train a multimodal large model based on the railway scene image and prompt word text pair to obtain a railway geological disaster recognition model; The disaster warning module is used to identify railway geological disasters based on the railway geological disaster identification model and real-time railway scene pictures, and to issue a warning when the existence of railway geological disasters is identified.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the railway geological disaster early warning method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the railway geological disaster early warning method described in any one of claims 1-6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the railway geological disaster early warning method according to any one of claims 1 to 6.

Images