Multi-modal large model-based elevator stop people trapping risk grade evaluation method

Abstract

The invention discloses a multi-modal large model-based elevator stopping people trapping risk grade evaluation method. A system comprises a DTU terminal, a communication server, an image analysis module and an automatic order placing module; the method comprises the following steps that S1, data uploaded by an elevator are received through a communication server, and images are collected when elevator stopping fault data are received; s2, the communication server sends an instruction to collect an elevator car image, and a DTU controls a camera in the elevator car to take a picture and transmits the picture back to the communication server; s3, the communication server receives the pictures in the lift car and then sends the pictures to a picture analysis module, and the picture analysis module calls the multi-mode large model to obtain picture description; s4, the image analysis module obtains the picture description and then sends the picture description to a risk assessment model for risk assessment, and a risk level is obtained; according to the method, the trapped person risk level is assessed through the multi-mode large model and the risk assessment model, then the processing priority is determined, the workload of customer service staff is relieved, the trapped person fault processing efficiency is improved, and elevator taking safety is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of elevator safety monitoring, and in particular relates to a method for assessing the risk level of people trapped in an elevator due to a stoppage based on a multi-modal large model. Background Art

[0002] With the development of cities and the improvement of living standards, elevators are used more and more frequently in life. As a vertical transportation tool, elevators may malfunction and stop due to various reasons during use. When the elevator stops, there may be passengers in the elevator car, and passengers trapped in the car are in danger (psychologically / physiologically) to a certain extent, and must be rescued in time to ensure the safety of passengers. In actual work, when an elevator stops, customer service personnel or sensor data will determine whether there are trapped people. If there are trapped people, priority must be given to ensure that the people are rescued in time. However, the risk level faced by normal adults and special groups (elderly / children / disabled people, etc.) is different.

[0003] Chinese invention patent CN118270617A discloses a method and system for early warning of elevator operation and entrapment risk based on deep learning, including the following steps: step S101, constructing a feature matrix; step S102, constructing a feature sequence. Step S103, inputting the feature matrix and feature sequence into the elevator risk assessment model, and the output value represents the elevator risk score within the preset prediction time period K; step S104, taking the first safety early warning measure according to the elevator risk score; the system includes a feature matrix construction module, which is used to set M image sampling points in the elevator car, machine room, and shaft, and collect the first image data of the elevator at a preset time interval n within the preset sampling time period m, and construct a feature matrix; a feature sequence construction module, which is used to collect the operation data and the second image data of the elevator at a preset time interval n within the preset collection time period m, and construct a feature sequence; an elevator risk score output module, which is used to input the feature matrix and feature sequence into the elevator risk assessment model, and the output value represents the elevator risk score within the preset prediction time period K. Risk scoring; a first safety warning measure generation module, which is used to take the first safety warning measure according to the elevator risk score; a manned image sequence generation module, which is used to extract the image inside the elevator car from the second image data at each time point in the second sequence to generate a manned image sequence; a maximum angle value calculation module, which is used to input the manned image sequence into the manned risk assessment model, and the output value represents the maximum angle value between the torso key points of the person in the manned image sequence and the elevator plane; a second safety warning measure generation module, which is used to take the second safety warning measure according to the maximum angle value; This solution uses a neural network model to extract features from the elevator image, and determines whether there is a dangerous situation through the angle of the limb key points, but this solution does not classify the risk level of entrapment, which is not convenient for subsequent customer service personnel to handle entrapment incidents.

[0004] Therefore, the present invention provides a method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model to solve the problems raised by the above-mentioned background technology. Summary of the invention

[0005] In response to the problems raised by the above background technology, the purpose of the present invention is to provide a method for assessing the risk level of entrapment due to elevator stop based on a multimodal large model. After an elevator stop failure occurs, the image inside the elevator car is automatically acquired, and the risk level of entrapment is assessed through the multimodal large model and the risk assessment model, and then the processing priority is determined, thereby reducing the workload of customer service personnel, improving the efficiency of handling entrapment failures, and ensuring elevator safety.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0007] A system for assessing the risk level of people trapped in an elevator due to a stoppage based on a multimodal large model, the system comprising a DTU terminal, a communication server, an image analysis module and an automatic ordering module, the DTU terminal being communicatively connected to the communication server, the communication server being communicatively connected to the image analysis module, and the image analysis module being communicatively connected to the automatic ordering module;

[0008] The DTU terminal is used to collect elevator operation status data and fault data in real time and send them to the communication server in real time. At the same time, the DTU terminal can perform image acquisition according to the communication server instructions;

[0009] The communication server is used to receive the elevator operation status data and fault data uploaded by the DTU terminal, and control the DTU terminal to collect the car image when receiving the elevator stop fault data;

[0010] The image analysis module is used to evaluate the risk level of the elevator car image and determine the processing priority;

[0011] The automatic order placement module is used to automatically send the fault handling order to the maintenance personnel.

[0012] The method for assessing the risk level of people trapped in an elevator due to elevator stop based on a multimodal large model includes the following steps:

[0013] S1: receiving data uploaded by the elevator through the communication server, and capturing images when elevator stop fault data is received;

[0014] S2: The communication server sends a command to collect car images, and the DTU controls the camera in the car to take pictures and transmit them back to the communication server;

[0015] S3: After receiving the photo of the car, the communication server sends it to the picture analysis module, and the picture analysis module calls the multimodal large model to obtain the picture description;

[0016] S4: the image analysis module obtains the picture description and sends it to the risk assessment model for risk assessment to obtain the risk level;

[0017] S5: The image analysis module determines the processing priority according to the risk level, and the higher the risk, the higher the priority;

[0018] S6: The automatic order placing module determines whether to place an order automatically according to the configuration.

[0019] It is further defined that when the communication server in S1 receives the elevator stop fault data, it first checks whether the DTU terminal supports the image acquisition function. If it does, it starts to acquire images. If it does not support image acquisition, the customer service staff confirms the elevator stop situation.

[0020] It is further defined that the risk level of the risk assessment model in S4 includes levels 0 to 3, where 0 means no risk, there is no one and no objects in the elevator car; 1 means low risk, there is no one but there are objects in the elevator car; 2 means medium risk, there is an adult male in the elevator car; 3 means high risk, there are elderly people, children, women or disabled people in the elevator car.

[0021] It is further defined that the multimodal large model in S3 is used to describe the collected images inside the car, so as to obtain the information inside the car described using natural language, and the natural language is input into the risk assessment model.

[0022] It is further defined that after receiving the natural language description text output by the multimodal large model, the risk assessment model performs word segmentation on the text, that is, divides a sentence into multiple word sequences, and sends the word sequences to the risk assessment model in sequence.

[0023] It is further defined that the model training process of the risk assessment model in S4 comprises the following steps:

[0024] S41: labeling pictures, labeling the collected car pictures, and labeling the risk level according to the situation of the car pictures;

[0025] S42: multimodal large model description, calling the multimodal large model to describe each car picture and save the corresponding description text;

[0026] S43: Integrate the description text and the label, and integrate the description text and the risk level of the image to form a training sample.

[0027] It is further defined that the model reasoning process of the risk assessment model in S4 comprises the following steps:

[0028] S411: receiving a picture, the image analysis module receives a car picture sent by the communication server;

[0029] S412: multi-modal large model description, calling the multi-modal large model to describe the car picture and outputting corresponding description text;

[0030] S413: Output the description text to the risk assessment model for risk level assessment.

[0031] It is further defined that if the automatic ordering module in S6 places an order automatically, the ordering process is automatically called to send the fault handling form to the maintenance personnel. If the order is not placed automatically, the order is placed after confirmation by the customer service personnel.

[0032] The method for assessing the risk level of people trapped in an elevator due to a stoppage based on a multimodal large model provided by the present invention has the following beneficial effects:

[0033] 1. The present invention utilizes the image description capability of a large multimodal model to achieve the conversion from image to natural language, thereby converting the image classification problem into a text classification problem, which is more intuitive and simple.

[0034] 2. The present invention utilizes open source or commercial multimodal large models, which not only saves the training cost of traditional large models, but also ensures that the description accuracy of the model itself is within an acceptable range, and can also continuously iterate and optimize the model.

[0035] 3. The present invention collects sample data during actual operation to continuously iteratively optimize the risk assessment model, thereby improving the model's accuracy and generalization ability.

[0036] 4. When the elevator of the present invention stops due to a fault, it automatically collects pictures of the car and performs a risk assessment. It can promptly confirm whether there are people trapped in the car and confirm the processing priority, which brings great convenience to customer service personnel. At the same time, it can automatically issue a fault handling form to maintenance personnel, reduce the workload of customer service personnel, improve the efficiency of handling trapped faults, shorten the fault handling time, ensure the safety of elevator passengers and reduce losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention can be further illustrated by the non-limiting examples given in the accompanying drawings;

[0038] Figure 1 It is a schematic diagram of the overall structure of an embodiment of a method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to the present invention;

[0039] Figure 2 It is a flowchart of the overall steps of an embodiment of a method for assessing the risk level of people trapped in an elevator due to a stoppage based on a multi-modal large model of the present invention;

[0040] Figure 3 It is a schematic diagram of the structure of a risk assessment module of an embodiment of a method for assessing the risk level of people trapped in an elevator due to a stoppage based on a multi-modal large model of the present invention;

[0041] Figure 4 It is a risk assessment model training flow chart of an embodiment of a method for assessing the risk level of people trapped in an elevator stop based on a multi-modal large model of the present invention;

[0042] Figure 5 It is a risk assessment model reasoning process diagram of an embodiment of a method for assessing the risk level of people trapped in an elevator stall based on a multi-modal large model of the present invention;

[0043] Figure 6 It is a system overall diagram of an embodiment of a method for assessing the risk level of people trapped in an elevator due to a stoppage based on a multi-modal large model of the present invention;

[0044] Figure 7 It is a text classification model diagram of an embodiment of a method for assessing the risk level of people trapped in an elevator stall based on a multimodal large model of the present invention;

[0045] Figure 8 It is an annotated diagram of a text classification model of an embodiment of a method for assessing the risk level of people trapped in an elevator stop based on a multimodal large model of the present invention;

[0046] Fig. 9 It is a text classification model reasoning process diagram of an embodiment of a method for assessing the risk level of people trapped in an elevator stall based on a multimodal large model of the present invention;

[0047] Fig.10 This is an example diagram of a car picture according to an embodiment of a method for assessing the risk level of entrapment due to elevator stop based on a multimodal large model of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments. The technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0050] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] like Figure 1As shown, the system of the risk level assessment method of elevator entrapment based on a multimodal large model of the present invention comprises a DTU terminal, a communication server, an image analysis module and an automatic ordering module, wherein the DTU terminal is communicatively connected to the communication server, the communication server is communicatively connected to the image analysis module, and the image analysis module is communicatively connected to the automatic ordering module;

[0052] The DTU terminal is used to collect elevator operation status data and fault data in real time and send them to the communication server in real time. At the same time, the DTU terminal can perform image acquisition according to the communication server instructions;

[0053] The communication server is used to receive the elevator operation status data and fault data uploaded by the DTU terminal, and control the DTU terminal to collect the car image when receiving the elevator stop fault data;

[0054] The image analysis module is used to evaluate the risk level of the elevator car image and determine the processing priority;

[0055] The automatic order placement module is used to automatically send the fault handling order to the maintenance personnel.

[0056] like Figure 2 As shown in the figure, the risk level assessment method of elevator entrapment based on a multimodal large model includes the following steps:

[0057] S1: receiving data uploaded by the elevator through the communication server, and capturing images when elevator stop fault data is received;

[0058] S2: The communication server sends a command to collect car images, and the DTU controls the camera in the car to take pictures and transmit them back to the communication server;

[0059] S3: After receiving the photo of the car, the communication server sends it to the picture analysis module, and the picture analysis module calls the multimodal large model to obtain the picture description;

[0060] S4: the image analysis module obtains the picture description and sends it to the risk assessment model for risk assessment to obtain the risk level;

[0061] S5: The image analysis module determines the processing priority according to the risk level, and the higher the risk, the higher the priority;

[0062] S6: The automatic order placing module determines whether to place an order automatically according to the configuration.

[0063] In the actual application of this embodiment, when the communication server in S1 receives the elevator stop fault data, it first checks whether the DTU terminal supports the image acquisition function. If it does, it starts to acquire images. If it does not support image acquisition, the customer service staff confirms the elevator stop situation.

[0064] In the actual application of this embodiment, the risk level of the risk assessment model in S4 includes 0 to 3 levels, 0 means no risk, there is no one and no objects in the elevator car; 1 means low risk, there is no one but there are objects in the elevator car; 2 means medium risk, there is an adult male in the elevator car; 3 means high risk, there are elderly people, children, women or disabled people in the elevator car.

[0065] In the practical application of this embodiment, the multimodal large model in S3 is used to describe the collected images in the car, so as to obtain the information in the car described in natural language, and the natural language is input into the risk assessment model. Since the training of the multimodal large model requires a large amount of materials and computing resources, and the cost-effectiveness of training a multimodal large model specifically for elevator scenes is very low, a commercial or open source multimodal large model can be used. Fig.10 As shown, the description given by the multimodal large model is: "The person in the photo is a male. He is wearing a black short-sleeved shirt, black shorts, and brown sandals. He is sitting in the elevator car with his hands on his knees, his legs straight, and his body leaning forward. There are yellow and black patterns on the floor of the elevator, which may be safety signs or decorations." It can be seen that the description of the multimodal large model is very accurate, which provides accurate input data for the subsequent risk assessment model.

[0066] In the actual application of this embodiment, after receiving the natural language description text output by the multimodal large model, the risk assessment model performs word segmentation on the text, that is, divides a sentence into multiple word sequences, and sends the word sequences to the risk assessment model in sequence.

[0067] like Figure 3 As shown, in the actual application of this embodiment, the risk assessment model in S4 includes 5 layers, the first layer is an embedding layer, and the embedding layer maps each word into a vector; specifically, during the model training process, the embedding vector will be continuously updated, and pre-trained word vectors (such as pre-trained word vectors of tencent-ailab or zhihu) can also be used. When using pre-trained word vectors, the embedding layer directly loads the pre-trained word vectors, and the embedding vectors will not be updated during the training process.

[0068] The second layer is a BiLSTM layer, i.e., a bidirectional long short-term memory layer, which is composed of a forward LSTM and a backward LSTM, and is used to better capture bidirectional semantic dependencies.

[0069] The third layer is a normalization layer, which is used to normalize the output data of the second layer (BiLSTM layer); the fourth layer is a ReLU activation function layer, which is used to activate the output data of the normalization layer.

[0070] The fifth layer is a Linear layer, i.e., a linear layer, which is used to perform a fully connected linear transformation on the output data of the activation function layer. The output data of the linear layer is the predicted probability of the risk level category; the risk level assessment is output by the activation function layer using the function LogSoftmax.

[0071] like Figure 4 As shown, in the practical application of this embodiment, the model training process of the risk assessment model in S4 includes the following steps:

[0072] S41: labeling pictures, labeling the collected car pictures, and labeling the risk level according to the situation of the car pictures;

[0073] S42: multimodal large model description, calling the multimodal large model to describe each car picture and save the corresponding description text;

[0074] S43: Integrate the description text and the label, and integrate the description text and the risk level of the image to form a training sample.

[0075] like Figure 5 As shown, in the practical application of this embodiment, the model reasoning process of the risk assessment model in S4 includes the following steps:

[0076] S411: receiving a picture, the image analysis module receives a car picture sent by the communication server;

[0077] S412: multi-modal large model description, calling the multi-modal large model to describe the car picture and outputting corresponding description text;

[0078] S413: Output the description text to the risk assessment model for risk level assessment.

[0079] In the actual application of this embodiment, if the automatic ordering module in S6 places an order automatically, the ordering process is automatically called to send the fault handling form to the maintenance personnel. If the order is not placed automatically, the order is placed after confirmation by the customer service staff.

[0080] like Figure 6 As shown in the figure, in the actual implementation of this system, the whole system is divided into image analysis service and front-end system. The front-end system refers to the existing elevator remote monitoring system. The image analysis service includes two processes: model training → deployment and inference. These two processes form a closed loop in the process of model iteration and optimization:

[0081] 1. By collecting images inside the elevator car after the elevator door is closed, these images can be simulated and collected in the elevator by themselves. After the images are collected, they are annotated according to the classification standards of the risk assessment model. Since the risk of entrapment is divided into 0-3 levels (0: no risk (no one and no objects in the car), 1: low risk (no one but objects in the car), 2: medium risk (adult male in the car); 3: high risk (old people / children / women / disabled people in the car)), there is no need to use special annotation software for annotation. You only need to create 4 folders, named 0, 1, 2, and 3 respectively, and classify the collected images into corresponding folders according to the risk level, that is, pictures with no one and no objects in the car are placed in the "0" folder, pictures with no one but objects in the car are placed in the "1" folder, pictures with adult males in the car are placed in the "2" folder, and pictures with old people / children / women / disabled people in the car are placed in the "3" folder.

[0082] 2. This embodiment does not classify images, but texts. Therefore, the multimodal large model needs to be called to describe the images in folders 0-3 and generate corresponding training samples. Assume that A0, B0, and C0 are in folder "0", and similarly A1, B1, and C1 are in folder "1", A2, B2, and C2 are in folder "2", and A3, B3, and C3 are in folder "3", as follows:

[0083]

[0084]

[0085] By calling the multimodal large model to describe all files, we get the corresponding description texts TA0, TB0, TC0, TA1, TB1, TC1, TA2, TB2, TC2, TA3, TB3, TC3, and then combine these description texts and classification values ​​into training sample files, as follows:

[0086] Classification label (risk level) Description text 0 TA0 0 TB0 0 TC0 1 TA1 1 TB1 1 TC1 2 TA2 2 TB2 2 TC2 3 TA3 3 TB3 3 TC3

[0087] For example:

[0088] The sample "0 There are no people or objects in the car, and the elevator is currently on the 3rd floor." means that the picture description is "There are no people or objects in the car, and the elevator is currently on the 3rd floor." The picture classification is "0", that is, the risk level is "0".

[0089] The sample "1 This is a photo of the interior of an elevator car. There is no one there, but there are some items, which may be boxes." means that the picture description is "This is a photo of the interior of an elevator car. There is no one there, but there are some items, which may be boxes." The picture classification is "1", that is, the risk level is "1".

[0090] At this point, this embodiment has converted the image classification problem into a text classification problem. The multimodal large model called by the image description can adopt a commercial model or an open source model. However, the multimodal large model used in the training process and the reasoning process should be kept as consistent as possible to ensure the accuracy of the reasoning results.

[0091] 3. After the sample data is generated, the sample data is divided into a training set, a validation set, and a test set in proportion. The training set is used to train the risk assessment model, the validation set is used to preliminarily evaluate the training effect of the risk assessment model, and the test set is used to confirm the final effect of the risk assessment model.

[0092] 4. The risk assessment model is implemented using a text classification model. The model structure is as follows: Figure 7 As shown; first, the Embedding layer uses 100, that is, the words are embedded (mapped) into a 100-dimensional vector. This is because the output of the pre-trained word vector used is 100-dimensional; the input of the BiLSTM layer is 100-dimensional, and the hidden layer is 128-dimensional. Since bidirectional LSTM is used, the actual output is (128*2)-dimensional; the input and output of the BatchNorm1d layer are both 256-dimensional; the input and output of the ReLU activation function layer are also 256-dimensional; the input of the Linear layer is 256-dimensional, and the output is 4-dimensional (that is, the probabilities of 4 categories); the input of the LogSoftmax activation function layer is 4-dimensional, and the output is 1-dimensional (that is, the category with the highest probability).

[0093] 5. Input data into the model in batches, use cross_entropy to calculate the loss for each batch, and perform backward propagation. Use the validation set to verify the model accuracy every 100 batches. After training, use the test set to test the model accuracy. Figure 8 As shown in the figure, in the forward process, all layers will participate, but in the backward process, the BiLSTM layer, BatchNorm1d layer and Linear layer will be adjusted according to the Loss (as shown in the red box in the figure), and other layers will not be adjusted. The Embedding layer will not be adjusted because it uses pre-trained word vectors.

[0094] 6. After the model training is completed and the accuracy reaches the design index, the model is deployed online to provide a calling interface for other systems (such as communication services). The model reasoning process is the forward process, such as Fig. 9 shown.

[0095] 7. After the system is deployed online, the images inside the car will be saved during each interface call, so that the images inside the car after people are actually trapped in the elevator can be continuously collected. These images can be used for continuous iterative optimization of the model. After a certain number of images are collected, repeat step 1 to continuously iterate and optimize the risk assessment model to improve the model accuracy and generalization ability.

[0096] 8. The front-end system calls the image analysis service interface to obtain the risk level assessment result, determines the priority based on the risk level assessment result, and determines whether to automatically place an order based on the front-end system configuration. If the order is automatically placed, the order process is automatically called to send the fault handling form to the maintenance personnel. If the order is not automatically placed, the order is placed after confirmation by the customer service staff.

[0097] In summary, this solution uses the image description capability of a large multimodal model to achieve the conversion from images to natural language, thereby converting the image classification problem into a text classification problem, which is more intuitive and simple. At the same time, the use of open source or commercial large multimodal models not only saves the training cost of the large model, but also ensures that the description accuracy of the model itself is within our acceptable range, and can also be continuously iterated and optimized with the model.

[0098] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A system for assessing the risk level of people trapped in an elevator due to elevator stop based on a multi-modal large model, characterized by: The system includes a DTU terminal, a communication server, an image analysis module and an automatic ordering module, wherein the DTU terminal is connected to the communication server for communication, the communication server is connected to the image analysis module for communication, and the image analysis module is connected to the automatic ordering module for communication; The DTU terminal is used to collect elevator operation status data and fault data in real time and send them to the communication server in real time. At the same time, the DTU terminal can perform image acquisition according to the communication server instructions; The communication server is used to receive the elevator operation status data and fault data uploaded by the DTU terminal, and control the DTU terminal to collect the car image when receiving the elevator stop fault data; The image analysis module is used to evaluate the risk level of the elevator car image and determine the processing priority; The automatic order placement module is used to automatically send the fault handling order to the maintenance personnel.

2. A risk level assessment method for elevator entrapment based on a multimodal large model, characterized in that: The steps include: S1: receiving data uploaded by the elevator through the communication server, and capturing images when elevator stop fault data is received; S2: The communication server sends a command to collect car images, and the DTU controls the camera in the car to take pictures and transmit them back to the communication server; S3: After receiving the photo of the car, the communication server sends it to the picture analysis module, and the picture analysis module calls the multimodal large model to obtain the picture description; S4: the image analysis module obtains the picture description and sends it to the risk assessment model for risk assessment to obtain the risk level; S5: The image analysis module determines the processing priority according to the risk level, and the higher the risk, the higher the priority; S6: The automatic order placing module determines whether to place an order automatically according to the configuration.

3. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to claim 2 is characterized in that: When the communication server in S1 receives the elevator stop fault data, it first checks whether the DTU terminal supports the image acquisition function. If it does, it starts to acquire images. If it does not support image acquisition, the customer service staff confirms the elevator stop situation.

4. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to claim 2 is characterized in that: The risk levels of the risk assessment model in S4 include 0 to 3, where 0 means no risk, that is, there is no one and no objects in the elevator car; 1 means low risk, that is, there is no one but there are objects in the elevator car; 2 means medium risk, that is, there is an adult male in the elevator car; 3 means high risk, that is, there are elderly people, children, women or disabled people in the elevator car.

5. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to claim 2 is characterized in that: The multimodal large model in S3 is used to describe the collected images inside the car, so as to obtain the information inside the car described in natural language, and the natural language is input into the risk assessment model.

6. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to claim 5 is characterized in that: After receiving the natural language description text output by the multimodal large model, the risk assessment model performs word segmentation on the text, that is, divides a sentence into multiple word sequences, and sends the word sequences to the risk assessment model in sequence.

7. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multimodal large model according to claim 2 is characterized in that: The model training process of the risk assessment model in S4 includes the following steps: S41: labeling pictures, labeling the collected car pictures, and labeling the risk level according to the situation of the car pictures; S42: multimodal large model description, calling the multimodal large model to describe each car picture and save the corresponding description text; S43: Integrate the description text and the label, and integrate the description text and the risk level of the image to form a training sample.

8. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to claim 2 is characterized in that: The model reasoning process of the risk assessment model in S4 includes the following steps: S411: receiving a picture, the image analysis module receives a car picture sent by the communication server; S412: multi-modal large model description, calling the multi-modal large model to describe the car picture and outputting corresponding description text; S413: Output the description text to the risk assessment model for risk level assessment.

9. The method for assessing the risk level of people trapped in an elevator due to an elevator stop based on a multi-modal large model according to claim 2 is characterized in that: If the automatic ordering module in S6 places an order automatically, the ordering process is automatically called to send the fault handling form to the maintenance personnel. If the order is not placed automatically, the order is placed after confirmation by the customer service personnel.

Images