Elevator fault accident recognition method, system and equipment based on intelligent image recognition and medium

Abstract

The invention provides an elevator fault accident recognition method, system and equipment based on intelligent image recognition and a medium, and belongs to the technical field of elevator fault diagnos.The method comprises the following steps that intelligent image sensors are assembled on the car roof of an elevator car and an elevator shaft tunnel in advance; image data in the elevator operation data collected by the intelligent image sensor is preprocessed, preliminary feature extraction is carried out, and a data set is constructed by combining the image data in the elevator operation data, the preliminarily extracted features and the elevator operation data of the known fault type; a deep learning model is constructed, the data set is used for training, and an elevator fault recognition model is obtained; and image data in elevator operation data collected by the elevator in real time and the preliminarily extracted features are input into an elevator fault recognition model, elevator operation faults are recognized, and an alarm is given. The intelligent image sensor is combined with the deep learning model to recognize the elevator fault, and the fault recognition efficiency and accuracy in the elevator running process are improved.

Description

Background Art

[0002] As an important vertical transportation tool in modern cities, the safety, reliability and operating efficiency of elevators are increasingly concerned. However, the complexity and diversity of elevator systems make elevator fault diagnosis extremely complicated. Traditional elevator fault diagnosis mainly relies on manual inspections and regular maintenance, which is not only inefficient but also difficult to fully cover all potential fault points. With the development of the Internet of Things, big data and artificial intelligence technologies, a new generation of elevator fault diagnosis has emerged and is developing in the direction of intelligence, automation and remoteness.

[0003] The new generation of elevator fault diagnosis is achieved through sensors, which requires the deployment of a large number of sensors in the elevator, such as speed sensors, temperature sensors, and pressure sensors. First of all, these sensors are not only complex to network, which increases the complexity and maintenance difficulty of the elevator system, but also have relatively high procurement costs and rear axle maintenance costs. Ultimately, it not only increases the economic burden on elevator manufacturers and operators, but also limits the widespread application of elevator fault diagnosis technology. Secondly, the performance of these sensors is greatly affected by environmental factors. In the elevator operating environment, changes in environmental factors such as temperature and humidity may cause sensor performance to decline, thereby affecting the accuracy of fault identification. In addition, problems such as sensor aging and damage may also cause inaccurate or lost data, further affecting the reliability of diagnosis.

[0004] This is a shortcoming of the prior art. Therefore, in view of the above-mentioned defects in the prior art, it is very necessary to provide an elevator fault accident identification method, system, equipment and medium based on intelligent image recognition. Summary of the invention

[0005] In view of the defects that the traditional elevator fault diagnosis relies on manual work and is inefficient, and the new method of using sensors increases the elevator operating cost, and the sensors are greatly affected by environmental factors, which affects the accuracy of elevator fault identification, the present invention provides an elevator fault accident identification method, system, equipment and medium based on intelligent image recognition to solve the above technical problems.

[0006] In a first aspect, the present invention provides an elevator fault accident identification method based on intelligent image recognition, comprising the following steps: S1. Pre-install intelligent image sensors on the top of the elevator car and the elevator shaft to collect elevator operation data; S2. Preprocess the image data in the elevator operation data collected by the intelligent image sensor, perform preliminary feature extraction, and construct a data set by combining the image data in the elevator operation data and the preliminary extracted features with the elevator operation data of known fault types; S3. Build a deep learning model and use the data set for training to obtain an elevator fault recognition model; S4. Input the image data and the initially extracted features in the elevator operation data collected in real time into the elevator fault recognition model to identify whether there is a fault in the elevator operation and the corresponding fault type, and issue an alarm according to the corresponding fault type. Furthermore, the specific steps of step S1 are as follows: S11. A smart camera device as a smart image sensor is installed on the top of the elevator car and the elevator shaft, and the smart camera device has a microphone module; S12. Establish a network connection between the smart camera device and the backend monitoring server; S13. The intelligent camera device collects elevator operation data in real time and sends it to the back-end monitoring server; the elevator operation data includes image data and sound data.

[0007] Furthermore, the specific steps of step S2 are as follows: S21. The back-end monitoring server performs noise reduction, contrast enhancement and grayscale processing on the image data in the elevator operation data, and uses a feature extraction algorithm to perform preliminary feature extraction to obtain the grayscale features of the image and the contour features of the objects in the image; S22. Acquire elevator fault image data from a public data set, wherein the elevator fault image data includes an elevator fire image and an elevator personnel fall image; S23. Mark the image data and grayscale features in the elevator operation data as elevator reverse fire data, extract grayscale features from the elevator fire images in the public data set after preprocessing, and mark them as elevator forward fire data to generate an elevator fire status data set; S24. Marking elevator fault operation data and elevator normal operation data according to the image data in the elevator operation data and the contour features of the objects in the image to generate an elevator operation recognition data set; S25. Mark normal images of personnel and images of personnel falling according to the image data in the elevator operation data and the contour features of the objects in the images, and generate an elevator personnel status data set.

[0008] Furthermore, the specific steps of step S21 are as follows: S211. Determine the noise reduction accuracy, select a filter window according to the noise reduction accuracy, and use the filter window to sequentially locate the image data of the image operation; S212. Sort the pixel values ​​in the positioning area by size, and use the middle pixel value as the pixel value of the entire positioning area to complete the noise reduction process; S213. Calculate the histogram of the image data of the elevator operation data, determine the mapping function according to the histogram, and then map the brightness value of the image data according to the mapping function to complete the contrast enhancement process; S214. Convert the image data of the elevator operation data from the color RGB space to the YUV space, and perform weighted average of the red component, the green component and the blue component according to the preset weight component to obtain the gray value, and complete the gray processing; S215. Use SIFT feature extraction algorithm to construct Gaussian pyramid, detect key points in the image by comparing pixel differences at different scales, and calculate the gradient direction histogram within the set area for each key point, and generate a vector descriptor of a specified dimension to characterize the grayscale features of the image; S216. Use the HOG feature extraction algorithm to calculate the gradient direction and size of each element in the image, and divide the image into set block sizes, count the gradient direction histogram of the pixels inside each block, and then aggregate a set number of blocks into clusters. The gradient direction histogram within each cluster is normalized and then connected in series to form a one-dimensional vector image feature descriptor to characterize the contour features of the image.

[0009] Furthermore, the specific steps of step S3 are as follows: S31. Construct an elevator fire recognition model, an elevator operation fault recognition model, and an elevator personnel safety accident recognition model, train the elevator fire recognition model through the elevator fire status data set, train the elevator operation fault recognition model through the elevator operation recognition data set, and train the elevator personnel safety accident recognition model through the elevator personnel status data set; S32. Integrate the trained elevator fire recognition model, elevator operation fault recognition model and elevator personnel safety accident recognition model into an elevator fault recognition model set, and deploy the elevator fault recognition model set to the back-end server.

[0010] Furthermore, the specific steps of step S31 are as follows: S311. The elevator fire recognition model based on convolutional neural network is as follows: Y fire = f fire ( W fire ⋅[ X fire , G fire ]+ b fire ); in, f fire Represents the activation function of the model and the processing of the output layer, X fire represents image data, G fire Represents the grayscale feature, Yfire Indicates the fire occurrence identification value, W fire represents the weights of all layers, b fire represents the bias of all layers; [ X fire , G fire ] represents the vector of image data and grayscale features; S312. The elevator operation fault recognition model is constructed based on the convolutional neural network as follows: Y run =f run (W run ⋅ [X run , C run ]+b run ) Among them, f run Represents the activation function of the model and the processing of the output layer, image data X run Its contour characteristics C run , weight parameter W run , bias parameter b run , Y run Indicates the identification value of elevator operation failure; [X run , C run ] represents the vector of image data and contour features; S313. The elevator personnel safety accident recognition model based on convolutional neural network is as follows: Y person =f person (W person ⋅ [X person , C person ]+b person ) Among them, f person represents the activation function of the model and the processing of the output layer, W person is the weight parameter, X person is the image data, b person is the bias parameter, Y person Indicates the identification value of personnel safety accidents; [X person ,C person ] represents the vector of image data and contour features; S314. Divide the elevator fire status data set into training data and test data, and use the training data to train the elevator fire recognition model, update the model parameters through the back propagation algorithm, and adjust the learning rate and batch block size after the minimum or maximum loss iteration, and then evaluate the accuracy and recall rate of the model through the test data to obtain the final elevator fire recognition model; S315. Divide the elevator operation identification data set into training data and test data according to the set ratio, use the training data to train the elevator operation fault identification model, and update the model parameters through the back propagation algorithm, verify it using the test set, adjust the learning rate and batch size, and then evaluate the model in combination with the evaluation index, and adjust the elevator operation fault identification model when the evaluation result does not meet the requirements, so as to obtain the optimal elevator operation fault identification model; S316. Divide the elevator personnel status data set into training data and test data, and use the training data to train the elevator personnel safety accident recognition model, update the model parameters through the back propagation algorithm, and adjust the learning rate and batch block size after the minimum or maximum loss iteration, and then evaluate the accuracy and recall rate of the model through the test data to obtain the final elevator personnel safety accident recognition model.

[0011] Furthermore, the specific steps of step S4 are as follows: S41. The back-end monitoring server inputs the image data and the grayscale features of the image in the real-time collected elevator operation data into the elevator fire recognition model in the elevator fault recognition model set, outputs the fire occurrence recognition value, and issues a fire alarm when the fire occurrence recognition value exceeds the set threshold; S42. The back-end monitoring server inputs the image data and image contour features in the real-time collected elevator operation data into the elevator operation fault recognition model in the elevator fault recognition model set, outputs the recognition value of each type of elevator operation fault, and issues a corresponding type of fault alarm when the recognition value of a certain type of elevator fault exceeds the threshold; S43. The back-end monitoring server inputs the image data and image contour features in the real-time collected elevator operation data into the elevator personnel safety accident recognition model in the elevator fault recognition set, outputs the recognition value of each type of personnel safety accident, and when the recognition value of a certain type of personnel safety accident exceeds the threshold, it combines the collected elevator operation sound data to issue a corresponding type of personnel safety alarm.

[0012] In a second aspect, the present invention provides an elevator fault accident identification system based on intelligent image recognition, comprising: The elevator operation data collection unit is used to install intelligent image sensors on the top of the elevator car and the elevator shaft in advance to collect elevator operation data; The elevator operation data preprocessing and feature extraction unit is used to preprocess the image data in the elevator operation data collected by the intelligent image sensor, perform preliminary feature extraction, and construct a data set by combining the image data in the elevator operation data and the preliminary extracted features with the elevator operation data of known fault types; The elevator fault identification model construction and training unit is used to construct a deep learning model and use the data set for training to obtain an elevator fault identification model; The elevator fault identification and alarm unit is used to input the image data and the initially extracted features in the elevator operation data collected in real time into the elevator fault identification model, identify whether there is a fault in the elevator operation and the corresponding fault type, and alarm according to the corresponding fault type.

[0013] In a third aspect, the present invention provides an electronic device, including a processor and a memory; The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method described in the first aspect above.

[0014] In a fourth aspect, the present invention provides a storage medium, The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect.

[0015] The beneficial effects of the present invention are: The elevator fault accident identification method, system, device and medium based on intelligent image recognition provided by the present invention identify elevator faults through intelligent image sensors combined with deep learning models, without the need for manual inspections, thereby improving the efficiency and accuracy of fault identification during elevator operation, avoiding the high cost and low recognition accuracy of traditional sensors, and improving the safety and reliability of elevators. The present invention can realize remote monitoring and diagnosis of elevator faults, and can quickly respond to and handle elevator faults.

[0016] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of the elevator fault accident identification method based on intelligent image recognition of the present invention.

[0019] Figure 2 It is a schematic diagram of an elevator fault accident identification system based on intelligent image recognition according to the present invention. DETAILED DESCRIPTION

[0020] The elevator fault accident identification method based on intelligent image recognition can be applied to one or more electronic devices, wherein the electronic device is an electronic device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0021] An electronic device may be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), an interactive network television (IPTV), etc. The network in which the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] See also Figure 1 The figure shows a specific implementation of an elevator fault accident identification method based on intelligent image recognition, which includes the following steps: S1. Pre-install intelligent image sensors on the top of the elevator car and the elevator shaft to collect elevator operation data; S2. Preprocess the image data in the elevator operation data collected by the intelligent image sensor, perform preliminary feature extraction, and construct a data set by combining the image data in the elevator operation data and the preliminary extracted features with the elevator operation data of known fault types; S3. Build a deep learning model and use the data set for training to obtain an elevator fault recognition model; S4. Input the image data and the initially extracted features in the elevator operation data collected in real time into the elevator fault recognition model to identify whether there is a fault in the elevator operation and the corresponding fault type, and issue an alarm according to the corresponding fault type.

[0024] In this embodiment, the specific steps of step S1 are as follows: S11. A smart camera device as a smart image sensor is installed on the top of the elevator car and the elevator shaft, and the smart camera device has a microphone module; S12. Establish a network connection between the smart camera device and the backend monitoring server; S13. The intelligent camera device collects elevator operation data in real time and sends it to the back-end monitoring server; the elevator operation data includes image data and sound data; The specific steps of step S2 are as follows: S21. The back-end monitoring server performs noise reduction, contrast enhancement and grayscale processing on the image data in the elevator operation data, and uses a feature extraction algorithm to perform preliminary feature extraction to obtain the grayscale features of the image and the contour features of the objects in the image; The specific steps of step S21 are as follows: S211. Determine the noise reduction accuracy, select a filter window according to the noise reduction accuracy, and use the filter window to sequentially locate the image data of the image operation; S212. Sort the pixel values ​​in the positioning area by size, and use the middle pixel value as the pixel value of the entire positioning area to complete the noise reduction process; S213. Calculate the histogram of the image data of the elevator operation data, determine the mapping function according to the histogram, and then map the brightness value of the image data according to the mapping function to complete the contrast enhancement process; S214. Convert the image data of the elevator operation data from the color RGB space to the YUV space, and perform weighted average of the red component, the green component and the blue component according to the preset weight component to obtain the gray value, and complete the gray processing; S215. Use SIFT feature extraction algorithm to construct Gaussian pyramid, detect key points in the image by comparing pixel differences at different scales, and calculate the gradient direction histogram within the set area for each key point, and generate a vector descriptor of a specified dimension to characterize the grayscale features of the image; S216. Calculate the gradient direction and size of each element in the image using the HOG feature extraction algorithm, divide the image into set block sizes, count the gradient direction histograms of the pixels in each block, aggregate a set number of blocks into clusters, normalize the gradient direction histograms in each cluster and connect them in series to form a one-dimensional vector image feature descriptor to characterize the contour features of the image; S22. Acquire elevator fault image data from a public data set, wherein the elevator fault image data includes an elevator fire image and an elevator personnel fall image; S23. Mark the image data and grayscale features in the elevator operation data as elevator reverse fire data, extract grayscale features from the elevator fire images in the public data set after preprocessing, and mark them as elevator forward fire data to generate an elevator fire status data set; S24. Marking the elevator fault operation data and the elevator normal operation data according to the image data in the elevator operation data and the contour features of the objects in the image, and generating an elevator operation recognition data set; for example, marking the contour features of the elevator door open or closed state, so as to identify the elevator door cannot be closed fault according to the contour features; S25. Marking normal images of personnel and images of personnel falling according to the contour features of the image data and objects in the image in the elevator operation data to generate an elevator personnel status data set; for example, marking the contour features of personnel in different states, so as to identify safety accidents of personnel falling according to the contour features; The specific steps of step S3 are as follows: S31. Construct an elevator fire recognition model, an elevator operation fault recognition model, and an elevator personnel safety accident recognition model, train the elevator fire recognition model through the elevator fire status data set, train the elevator operation fault recognition model through the elevator operation recognition data set, and train the elevator personnel safety accident recognition model through the elevator personnel status data set; The specific steps of step S31 are as follows: S311. The elevator fire recognition model based on convolutional neural network is as follows: Y fire = f fire ( W fire ⋅[ X fire , G fire ]+ b fire ); in, f fire Represents the activation function of the model and the processing of the output layer, X fire represents image data, G fire Represents the grayscale feature, Y fire Indicates the fire occurrence identification value, W fire represents the weights of all layers, b fire represents the bias of all layers; [ X fire , G fire] represents the vector of image data and grayscale features; All layers include convolutional layers, pooling layers, fully connected layers, and output layers except the input layer; S312. The elevator operation fault recognition model is constructed based on the convolutional neural network as follows: Y run =f run (W run ⋅ [X run , C run ]+b run ) Among them, f run Represents the activation function of the model and the processing of the output layer, image data X run Its contour characteristics C run , weight parameter W run , bias parameter b run , Y run Indicates the identification value of elevator operation failure; [X run , C run ] represents the vector of image data and contour features; S313. The elevator personnel safety accident recognition model based on convolutional neural network is as follows: Y person =f person (W person ⋅ [X person , C person ]+b person ) Among them, f person represents the activation function of the model and the processing of the output layer, W person is the weight parameter, X person is the image data, b person is the bias parameter, Y person Indicates the identification value of personnel safety accidents; [X person ,C person ] represents the vector of image data and contour features; S314. Divide the elevator fire status data set into training data and test data, and use the training data to train the elevator fire recognition model, update the model parameters through the back propagation algorithm, and adjust the learning rate and batch block size after the minimum or maximum loss iteration, and then evaluate the accuracy and recall rate of the model through the test data to obtain the final elevator fire recognition model; S315. Divide the elevator operation identification data set into training data and test data according to the set ratio, use the training data to train the elevator operation fault identification model, and update the model parameters through the back propagation algorithm, verify it using the test set, adjust the learning rate and batch size, and then evaluate the model in combination with the evaluation index, and adjust the elevator operation fault identification model when the evaluation result does not meet the requirements, so as to obtain the optimal elevator operation fault identification model; S316. Divide the elevator personnel status data set into training data and test data, and use the training data to train the elevator personnel safety accident recognition model, update the model parameters through the back propagation algorithm, and adjust the learning rate and batch block size after the minimum or maximum loss iteration, and then evaluate the accuracy and recall rate of the model through the test data to obtain the final elevator personnel safety accident recognition model; S32. Integrate the trained elevator fire recognition model, elevator operation fault recognition model, and elevator personnel safety accident recognition model into an elevator fault recognition model set, and deploy the elevator fault recognition model set to the backend server; The specific steps of step S4 are as follows: S41. The back-end monitoring server inputs the image data and the grayscale features of the image in the real-time collected elevator operation data into the elevator fire recognition model in the elevator fault recognition model set, outputs the fire occurrence recognition value, and issues a fire alarm when the fire occurrence recognition value exceeds the set threshold; S42. The back-end monitoring server inputs the image data and image contour features in the real-time collected elevator operation data into the elevator operation fault recognition model in the elevator fault recognition model set, outputs the recognition value of each type of elevator operation fault, and issues a corresponding type of fault alarm when the recognition value of a certain type of elevator fault exceeds the threshold; S43. The back-end monitoring server inputs the image data and image contour features in the real-time collected elevator operation data into the elevator personnel safety accident recognition model in the elevator fault recognition set, outputs the recognition value of each type of personnel safety accident, and when the recognition value of a certain type of personnel safety accident exceeds the threshold, it combines the collected elevator operation sound data to issue a corresponding type of personnel safety alarm.

[0025] It should be understood that the order of execution of the steps in the above embodiment does not mean that the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0026] The following is an embodiment of an elevator fault accident identification system based on intelligent image recognition provided by an embodiment of the present disclosure. This system and the elevator fault accident identification method based on intelligent image recognition in the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the elevator fault accident identification system based on intelligent image recognition, please refer to the embodiment of the elevator fault accident identification method based on intelligent image recognition.

[0027] like Figure 2 As shown, the system includes: The elevator operation data collection unit is used to install intelligent image sensors on the top of the elevator car and the elevator shaft in advance to collect elevator operation data; The elevator operation data preprocessing and feature extraction unit is used to preprocess the image data in the elevator operation data collected by the intelligent image sensor, perform preliminary feature extraction, and construct a data set by combining the image data in the elevator operation data and the preliminary extracted features with the elevator operation data of known fault types; The elevator fault identification model construction and training unit is used to construct a deep learning model and use the data set for training to obtain an elevator fault identification model; The elevator fault identification and alarm unit is used to input the image data and the initially extracted features in the elevator operation data collected in real time into the elevator fault identification model, identify whether there is a fault in the elevator operation and the corresponding fault type, and alarm according to the corresponding fault type.

[0028] The elevator fault accident identification method based on intelligent image recognition provided by the present invention is a combination of the modules and algorithm steps of each example described in the embodiments disclosed herein, and can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0029] Those skilled in the art will appreciate that the various aspects of the elevator fault accident identification method based on intelligent image recognition provided by the present invention can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, which can be collectively referred to as "circuit", "module" or "system" here.

[0030] In the non-transitory computer-readable storage medium storing the elevator fault accident identification method based on intelligent image recognition, it can be but not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0031] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A method for identifying elevator fault accidents based on intelligent image recognition, characterized in that: The steps include: S1. Pre-install intelligent image sensors on the top of the elevator car and the elevator shaft to collect elevator operation data; S2. Preprocess the image data in the elevator operation data collected by the intelligent image sensor, perform preliminary feature extraction, and construct a data set by combining the image data in the elevator operation data and the preliminary extracted features with the elevator operation data of known fault types; S3. Build a deep learning model and use the data set for training to obtain an elevator fault recognition model; S4. Input the image data and the initially extracted features in the elevator operation data collected in real time into the elevator fault recognition model to identify whether there is a fault in the elevator operation and the corresponding fault type, and issue an alarm according to the corresponding fault type.

2. The elevator fault accident identification method based on intelligent image recognition according to claim 1, characterized in that: The specific steps of step S1 are as follows: S11. A smart camera device as a smart image sensor is installed on the top of the elevator car and the elevator shaft, and the smart camera device has a microphone module; S12. Establish a network connection between the smart camera device and the backend monitoring server; S13. The intelligent camera device collects elevator operation data in real time and sends it to the back-end monitoring server; the elevator operation data includes image data and sound data.

3. The elevator fault accident identification method based on intelligent image recognition according to claim 2, characterized in that: The specific steps of step S2 are as follows: S21. The back-end monitoring server performs noise reduction, contrast enhancement and grayscale processing on the image data in the elevator operation data, and uses a feature extraction algorithm to perform preliminary feature extraction to obtain the grayscale features of the image and the contour features of the objects in the image; S22. Acquire elevator fault image data from a public data set, wherein the elevator fault image data includes an elevator fire image and an elevator personnel fall image; S23. Mark the image data and grayscale features in the elevator operation data as elevator reverse fire data, extract grayscale features from the elevator fire images in the public data set after preprocessing, and mark them as elevator forward fire data to generate an elevator fire status data set; S24. Marking elevator fault operation data and elevator normal operation data according to the image data in the elevator operation data and the contour features of the objects in the image to generate an elevator operation recognition data set; S25. Mark normal images of personnel and images of personnel falling according to the image data in the elevator operation data and the contour features of the objects in the images, and generate an elevator personnel status data set.

4. The elevator fault accident identification method based on intelligent image recognition according to claim 3, characterized in that: The specific steps of step S21 are as follows: S211. Determine the noise reduction accuracy, select a filter window according to the noise reduction accuracy, and use the filter window to sequentially locate the image data of the image operation; S212. Sort the pixel values ​​in the positioning area by size, and use the middle pixel value as the pixel value of the entire positioning area to complete the noise reduction process; S213. Calculate the histogram of the image data of the elevator operation data, determine the mapping function according to the histogram, and then map the brightness value of the image data according to the mapping function to complete the contrast enhancement process; S214. Convert the image data of the elevator operation data from the color RGB space to the YUV space, and perform weighted average of the red component, the green component and the blue component according to the preset weight component to obtain the gray value, and complete the gray processing; S215. Use SIFT feature extraction algorithm to construct Gaussian pyramid, detect key points in the image by comparing pixel differences at different scales, and calculate the gradient direction histogram within the set area for each key point, and generate a vector descriptor of a specified dimension to characterize the grayscale features of the image; S216. Use the HOG feature extraction algorithm to calculate the gradient direction and size of each element in the image, and divide the image into set block sizes, count the gradient direction histogram of the pixels inside each block, and then aggregate a set number of blocks into clusters. The gradient direction histogram within each cluster is normalized and then connected in series to form a one-dimensional vector image feature descriptor to characterize the contour features of the image.

5. The elevator fault accident identification method based on intelligent image recognition according to claim 3, characterized in that: The specific steps of step S3 are as follows: S31. Construct an elevator fire recognition model, an elevator operation fault recognition model, and an elevator personnel safety accident recognition model, train the elevator fire recognition model through the elevator fire status data set, train the elevator operation fault recognition model through the elevator operation recognition data set, and train the elevator personnel safety accident recognition model through the elevator personnel status data set; S32. Integrate the trained elevator fire recognition model, elevator operation fault recognition model and elevator personnel safety accident recognition model into an elevator fault recognition model set, and deploy the elevator fault recognition model set to the back-end server.

6. The elevator fault accident identification method based on intelligent image recognition according to claim 5, characterized in that: The specific steps of step S31 are as follows: S311. The elevator fire recognition model based on convolutional neural network is as follows: Y fire = f fire ( W fire ⋅[ X fire ， G fire ]+ b fire )； in, f fire Represents the activation function of the model and the processing of the output layer, X fire represents image data, G fire Represents the grayscale feature, Y fire Indicates the fire occurrence identification value, W fire represents the weights of all layers, b fire represents the bias of all layers; [ X fire , G fire ] represents the vector of image data and grayscale features; S312. The elevator operation fault recognition model is constructed based on the convolutional neural network as follows: Y run =f run (W run ⋅ [X run ,C run ]+b run ) Among them, f run Represents the activation function of the model and the processing of the output layer, image data X run Its contour characteristics C run , weight parameter W run , bias parameter b run , Y run Indicates the identification value of elevator operation failure; [X run , C run ] represents the vector of image data and contour features; S313. The elevator personnel safety accident recognition model based on convolutional neural network is as follows: Y person =f person (W person ⋅ [X person ,C person ]+b person ) Among them, f person represents the activation function of the model and the processing of the output layer, W person is the weight parameter, X person is the image data, b person is the bias parameter, Y person Indicates the identification value of personnel safety accidents; [X person ,C person ] represents the vector of image data and contour features; S314. Divide the elevator fire status data set into training data and test data, and use the training data to train the elevator fire recognition model, update the model parameters through the back propagation algorithm, and adjust the learning rate and batch block size after the minimum or maximum loss iteration, and then evaluate the accuracy and recall rate of the model through the test data to obtain the final elevator fire recognition model; S315. Divide the elevator operation identification data set into training data and test data according to the set ratio, use the training data to train the elevator operation fault identification model, and update the model parameters through the back propagation algorithm, verify it using the test set, adjust the learning rate and batch size, and then evaluate the model in combination with the evaluation index, and adjust the elevator operation fault identification model when the evaluation result does not meet the requirements, so as to obtain the optimal elevator operation fault identification model; S316. Divide the elevator personnel status data set into training data and test data, and use the training data to train the elevator personnel safety accident recognition model, update the model parameters through the back propagation algorithm, and adjust the learning rate and batch block size after the minimum or maximum loss iteration, and then evaluate the accuracy and recall rate of the model through the test data to obtain the final elevator personnel safety accident recognition model.

7. The elevator fault accident identification method based on intelligent image recognition according to claim 5, characterized in that: The specific steps of step S4 are as follows: S41. The back-end monitoring server inputs the image data and the grayscale features of the image in the real-time collected elevator operation data into the elevator fire recognition model in the elevator fault recognition model set, outputs the fire occurrence recognition value, and issues a fire alarm when the fire occurrence recognition value exceeds the set threshold; S42. The back-end monitoring server inputs the image data and image contour features in the real-time collected elevator operation data into the elevator operation fault recognition model in the elevator fault recognition model set, outputs the recognition value of each type of elevator operation fault, and issues a corresponding type of fault alarm when the recognition value of a certain type of elevator fault exceeds the threshold; S43. The back-end monitoring server inputs the image data and image contour features in the real-time collected elevator operation data into the elevator personnel safety accident recognition model in the elevator fault recognition set, outputs the recognition value of each type of personnel safety accident, and when the recognition value of a certain type of personnel safety accident exceeds the threshold, it combines the collected elevator operation sound data to issue a corresponding type of personnel safety alarm.

8. An elevator fault accident identification system based on intelligent image recognition, characterized in that: include: The elevator operation data collection unit is used to install intelligent image sensors on the top of the elevator car and the elevator shaft in advance to collect elevator operation data; The elevator operation data preprocessing and feature extraction unit is used to preprocess the image data in the elevator operation data collected by the intelligent image sensor, perform preliminary feature extraction, and construct a data set by combining the image data in the elevator operation data and the preliminary extracted features with the elevator operation data of known fault types; The elevator fault identification model construction and training unit is used to construct a deep learning model and use the data set for training to obtain an elevator fault identification model; The elevator fault identification and alarm unit is used to input the image data and the initially extracted features in the elevator operation data collected in real time into the elevator fault identification model, identify whether there is a fault in the elevator operation and the corresponding fault type, and alarm according to the corresponding fault type.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in any one of claims 1 to 7.

Images