Elevator control cabinet monitoring method based on visible light and infrared image fusion

Abstract

The invention discloses an elevator control cabinet monitoring method based on visible light and infrared image fusion, which comprises the following steps: acquiring and optimizing an initial elevator control cabinet image to obtain an optimized elevator control cabinet image; registering the visible light camera and the infrared camera to obtain a camera extrinsic parameter matrix; acquiring an elevator control cabinet component data set, and constructing a target detection network model based on the elevator control cabinet component data set; the optimized elevator control cabinet image is input into a target detection network model, the position of a visible light elevator control cabinet component is obtained, the position of the elevator control cabinet component is mapped to an elevator control cabinet infrared image through a camera external parameter matrix, and the temperature of the corresponding elevator control cabinet component is calculated based on the infrared image; and temperature matching and real-time monitoring of elevator control cabinet components are realized. Abnormity of the elevator control cabinet can be found in time, and safe operation of an elevator is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator safety monitoring, and in particular to an elevator control cabinet monitoring method based on visible light and infrared image fusion. Background Art

[0002] With the rapid development of urbanization and the improvement of residents' living standards, elevators have become an indispensable means of vertical transportation in modern buildings, significantly improving travel efficiency and playing a key role in social and economic activities.

[0003] The elevator control cabinet is one of the core components of the elevator. It is responsible for controlling the operation and various safety functions of the elevator. It contains key components such as control circuit boards, inverters, relays, power supplies, etc., which together ensure the safe and efficient operation of the elevator. When a component fails, it will seriously affect the safe operation of the elevator.

[0004] The control circuit board usually contains components such as microprocessors, storage chips, power chips, etc., and the machine room environment is relatively complex. Due to vibration, overload, dust and other reasons, some components may be damaged, resulting in abnormal heating. In addition, the terminals, connectors, contactors and other components in the control cabinet may also cause abnormal heating due to loose connections, moisture and other reasons. The existing method usually performs functional testing on the elevator control cabinet circuit board before leaving the factory, and the on-site inspection of the elevator control cabinet is still carried out through manual inspection during maintenance time. It is impossible to monitor the elevator control cabinet throughout the entire cycle, and it is difficult to detect abnormal phenomena in the elevator control cabinet in time. In view of the characteristic of abnormal heating when the key components of the elevator control cabinet fail, this patent proposes an elevator control cabinet monitoring method based on the fusion of visible light and infrared images to complete the real-time monitoring of the temperature of the elevator control cabinet components, so as to achieve timely feedback on the elevator control cabinet failure and protect the safety of the elevator. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an elevator control cabinet monitoring method based on the fusion of visible light and infrared images, so as to achieve timely feedback on elevator control cabinet failures and protect elevator safety.

[0006] To achieve the above object, the present invention provides an elevator control cabinet monitoring method based on visible light and infrared image fusion, comprising:

[0007] Collecting and optimizing an initial elevator control cabinet image to obtain an optimized elevator control cabinet image, wherein the initial elevator control cabinet image includes an initial elevator control cabinet visible light image and an initial elevator control cabinet infrared image;

[0008] Align the visible light camera and the infrared camera to obtain the camera extrinsic parameter matrix;

[0009] Acquire an elevator control cabinet component data set, and build a target detection network model based on the elevator control cabinet component data set;

[0010] The optimized elevator control cabinet image is input into the target detection network model to obtain the position of the visible light elevator control cabinet components, and the position of the elevator control cabinet components is mapped to the elevator control cabinet infrared image through the camera extrinsic parameter matrix. The temperature of the corresponding elevator control cabinet components is calculated based on the infrared image to achieve elevator control cabinet component temperature matching and real-time monitoring.

[0011] Preferably, collecting and optimizing the initial elevator control cabinet image includes:

[0012] The initial visible light image of the elevator control cabinet and the initial infrared image of the elevator control cabinet are collected respectively, and the initial visible light image of the elevator control cabinet is subjected to distortion correction to obtain an optimized visible light image of the elevator control cabinet. The initial infrared image of the elevator control cabinet is subjected to distortion correction and bilateral filtering to obtain an optimized infrared image of the elevator control cabinet.

[0013] Preferably, the process of performing the optimization includes:

[0014] Calibrate the visible light camera and the infrared camera to obtain the visible light camera internal parameter matrix and the infrared camera internal parameter matrix, wherein the visible light camera is used to collect the visible light image of the elevator control cabinet, and the infrared camera is used to collect the infrared image of the elevator control cabinet;

[0015] Using the visible light camera internal parameter matrix to perform distortion correction on the initial elevator control cabinet visible light image to obtain an elevator control cabinet visible light image;

[0016] The infrared camera internal parameter matrix is ​​used to perform distortion correction on the initial elevator control cabinet infrared image, the distortion-corrected elevator control cabinet infrared image is output, and bilateral filtering is used to perform noise reduction on the image to obtain the elevator control cabinet infrared image, thereby completing image optimization.

[0017] Preferably, calibrating the visible light camera and the infrared camera includes:

[0018] Collect a number of visible light calibration plate images and infrared calibration plate images respectively, solve all the center coordinates in each of the visible light calibration plate images and the infrared calibration plate images respectively, and use Zhang's calibration method to obtain the visible light camera internal parameter matrix and the infrared camera internal parameter matrix to complete the calibration;

[0019] When the visible light camera is used to collect the image of the visible light calibration plate, the calibration plate is directly photographed; when the infrared camera is used to collect the image of the infrared calibration plate, the LED light is powered on first, and then the calibration plate is photographed when the heating is stable.

[0020] Preferably, the visible light camera and the infrared camera are registered to obtain the camera extrinsic parameter matrix, including:

[0021] Collecting a visible light registration image and an infrared registration image respectively, solving the center coordinates of circles in the visible light registration image and the infrared registration image, and selecting a number of center coordinates as feature points;

[0022] The eight-point method is used to solve the feature points to obtain the camera basic matrix;

[0023] The basic matrix is ​​decomposed based on SVD to obtain a rotation matrix and a translation vector, thereby completing camera registration; wherein the camera extrinsic parameter matrix includes the rotation matrix and the translation vector.

[0024] Preferably, obtaining an elevator control cabinet component data set includes:

[0025] Based on the visible light camera, several visible light images of elevator control cabinets are collected to form a data set, and target annotation is performed on several visible light images of elevator control cabinets to obtain data set labels;

[0026] The categories of the data set labels include control circuit boards, microprocessors, contactors, and relays; and the label information of the data set labels includes the category, x coordinate, y coordinate, w width, and h height information of each target.

[0027] Preferably, the target detection network model is trained by a training set and verified by a verification set, and both the training set and the verification set are selected from the elevator control cabinet component data set;

[0028] The target detection network model includes an input layer, a hidden layer and an output layer, wherein the input layer is used to input a visible light image of an elevator control cabinet; the hidden layer is composed of a plurality of convolutional layers with different convolutional kernel sizes and different numbers of convolutional kernels; and the output layer is used to output the positions of all visible light elevator control cabinet components in the visible light image of the elevator control cabinet.

[0029] Preferably, the process of training the target detection network model includes:

[0030] S1. Input the elevator control cabinet training set into the input layer to form a tensor feature map, forward propagate the tensor feature map, extract features after passing through the hidden layer, process the extracted features based on the output layer, and output the visible light elevator control cabinet component position;

[0031] S2, inputting the visible light elevator control cabinet component position and the elevator control cabinet test set into the loss function, calculating the loss value, and back-propagating along the gradient direction of the loss function through the loss value to update the model weight parameters;

[0032] S3. Repeat S1-S2 until the average accuracy is greater than the threshold, and obtain the optimal target detection model to be determined, that is, the target detection network model.

[0033] Preferably, the position of the elevator control cabinet component is mapped to the elevator control cabinet infrared image through the camera external parameter matrix, and the temperature of the corresponding elevator control cabinet component is calculated based on the infrared image to achieve the temperature matching and real-time monitoring of the elevator control cabinet component, including:

[0034] The visible light elevator control cabinet component position is multiplied by the rotation matrix and added with the translation vector to obtain the infrared elevator control cabinet component position, and the average and peak values ​​of the temperatures corresponding to all pixel points in each infrared elevator control cabinet component position are solved to complete the elevator control cabinet component temperature matching, and the temperature of each elevator control cabinet component is monitored in real time. When the temperature is greater than the set value, an abnormal warning is issued.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] The present invention realizes the registration of visible light camera and infrared camera based on the eight-point method and SVD decomposition method, and establishes and trains the optimal target detection network model by constructing the elevator control cabinet component data set, thereby realizing the component position recognition of the elevator control cabinet, and further realizing the temperature matching and real-time monitoring of the elevator control cabinet components according to the camera extrinsic parameter matrix, so as to timely detect the abnormality of the elevator control cabinet and ensure the safe operation of the elevator. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0038] Figure 1 This is a schematic diagram of the structure of the elevator control cabinet image acquisition device according to an embodiment of the present invention, wherein 1-fixing hole, 2-bottom plate, 3-fill light, 4-camera bracket, 5-visible light camera, 6-infrared camera;

[0039] Figure 2 This is a flow chart of an elevator control cabinet monitoring method based on visible light and infrared image fusion according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a calibration plate according to an embodiment of the present invention;

[0041] Figure 4 A process data diagram for constructing an elevator control cabinet component data set according to an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of the target detection model structure and the target detection model training process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] The present invention proposes an elevator control cabinet monitoring method based on the fusion of visible light and infrared images. Figure 2 ,include:

[0046] Collecting and optimizing an initial elevator control cabinet image to obtain an optimized elevator control cabinet image, wherein the initial elevator control cabinet image includes an initial elevator control cabinet visible light image and an initial elevator control cabinet infrared image;

[0047] Align the visible light camera and the infrared camera to obtain the camera extrinsic parameter matrix;

[0048] Obtain an elevator control cabinet component dataset, and build a target detection network model based on the elevator control cabinet component dataset;

[0049] The optimized elevator control cabinet image is input into the target detection network model to obtain the position of the visible light elevator control cabinet components, and the position of the elevator control cabinet components is mapped to the elevator control cabinet infrared image through the camera extrinsic parameter matrix. The temperature of the corresponding elevator control cabinet components is calculated based on the infrared image to achieve elevator control cabinet component temperature matching and real-time monitoring.

[0050] This embodiment realizes the component position recognition of the elevator control cabinet, and further realizes the temperature matching and real-time monitoring of the elevator control cabinet components according to the camera external parameter matrix, so that the abnormality of the elevator control cabinet can be discovered in time, ensuring the safe operation of the elevator.

[0051] Furthermore, the initial elevator control cabinet image is collected and optimized, including:

[0052] The initial visible light image of the elevator control cabinet and the initial infrared image of the elevator control cabinet are respectively collected by the elevator control cabinet image acquisition device, and the initial visible light image of the elevator control cabinet is subjected to distortion correction processing to obtain the optimized visible light image of the elevator control cabinet. The initial infrared image of the elevator control cabinet is subjected to distortion correction and bilateral filtering processing to obtain the optimized infrared image of the elevator control cabinet.

[0053] Furthermore, the elevator control cabinet image acquisition device (such as Figure 1 ) includes a fill light 3, a camera bracket 4, a visible light camera 5 and an infrared camera 6;

[0054] The elevator control cabinet image acquisition device is fixed on the switch door of the elevator control cabinet through the fixing hole 1 and the bottom plate 2; the visible light camera 5 and the infrared camera 6 are packaged and fixed on the camera bracket 4, and the camera bracket 4 is fixed on the bottom plate 2;

[0055] The fill light 3 is used to supplement the brightness inside the elevator control cabinet, the visible light camera 5 is used to collect the initial visible light image of the elevator control cabinet, and the infrared camera 6 is used to collect the initial infrared image of the elevator control cabinet.

[0056] Furthermore, the optimization process includes:

[0057] Calibrate the visible light camera and the infrared camera to obtain the visible light camera intrinsic parameter matrix and the infrared camera intrinsic parameter matrix;

[0058] Input the initial elevator control cabinet visible light image and the visible light camera internal parameter matrix into the undistort() function in OpenCV, output the distortion-corrected elevator control cabinet visible light image, and complete the image optimization;

[0059] The initial elevator control cabinet infrared image and the infrared camera internal parameter matrix are input into the undistort() function in OpenCV, the elevator control cabinet infrared image after distortion correction is output, and the image is subjected to noise reduction processing by using bilateral filtering to complete image optimization.

[0060] Specifically, bilateral filtering is a nonlinear filtering method that is a compromise between the spatial proximity of an image and the similarity of pixel values. It also takes into account the spatial difference and intensity difference of pixels to achieve edge-preserving denoising. The specific process is as follows: a 5×5 window is established with a certain pixel point in the image as the center, and the spatial domain G of each pixel point in the window is traversed and calculated. σs With pixel domain G σr , and G σs With G σr Multiply to get the weight W of each pixel pThen, the sum of the weights of all pixels is divided by the sum of the products of the weights of all pixels and their pixel values ​​to obtain the new pixel value after filtering of the pixel. All pixels of the image are traversed to complete bilateral filtering.

[0061] Among them, calculate G σs With G σr The formula is as follows:

[0062]

[0063] Where i and j are the coordinates of the pixel value to be processed, k and l are the coordinates of the pixel in the window, σs is 5, and σr is 20.

[0064] Furthermore, the visible light camera and the infrared camera are calibrated, including:

[0065] Collect several visible light calibration plate images and infrared calibration plate images respectively, use the imfindcircles() function in MATLAB software to solve the coordinates of all the center points in each of the visible light calibration plate images and the infrared calibration plate images respectively, and input all the center point coordinates into the calibrateCamera() function in OpenCV, and use Zhang's calibration method to obtain the visible light camera intrinsic parameter matrix and the infrared camera intrinsic parameter matrix to complete the calibration;

[0066] When the visible light calibration plate image is collected by the visible light camera, the calibration plate is photographed directly; when the infrared calibration plate image is collected by the infrared camera, the LED light is powered on first, and then the calibration plate is photographed when the heating is stable.

[0067] Specifically, the specific process of visible light camera calibration is as follows: in this embodiment, a visible light camera is used to collect 20 visible light calibration plate images, and the imfindcircles() function in the MATLAB software is used to solve the coordinates of all the center points in each visible light calibration plate image, and further all the center point coordinates are input into the calibrateCamera() function in OpenCV, and the visible light camera intrinsic parameter matrix is ​​solved through Zhang's calibration method; the process of infrared camera calibration is consistent with the process of visible light camera calibration.

[0068] In this embodiment, the calibration plate is a rectangle of 210 mm × 297 mm. Figure 3 As shown, there are 6 rows and 9 columns of circular holes, the diameter of the circular holes is 20 mm, and the spacing is 30 mm. In addition, LED lights are set in the circular holes as heating elements, and calibration paper is pasted on the surface of the calibration plate. The positions corresponding to the circular holes are black, and the other positions are white.

[0069] Furthermore, the visible light camera and the infrared camera are registered to obtain the camera extrinsic parameter matrix, including:

[0070] A visible light registration image and an infrared registration image are collected respectively, and the imfindcircles() function in MATLAB software is used to solve the center coordinates of the circles in the visible light registration image and the infrared registration image, and a number of center coordinates are selected as feature points;

[0071] The eight-point method is used to solve the feature points to obtain the camera basic matrix;

[0072] The basic matrix is ​​decomposed based on SVD to obtain a rotation matrix and a translation vector, thereby completing camera registration; wherein the camera extrinsic parameter matrix includes the rotation matrix and the translation vector.

[0073] Specifically, the process of camera registration is:

[0074] S21: Acquire registration images;

[0075] S22: Collect feature points;

[0076] S23: using the eight-point method to solve the basic matrix F;

[0077] S24: Decompose the basic matrix F through SVD to solve the camera extrinsic matrix.

[0078] In S21, the process of collecting the registration image is: keep the relative positions of the visible light camera, the infrared camera, and the calibration plate unchanged, use the visible light camera to collect a visible light registration image, power on the LED light, and after the heat is stable, use the infrared camera to collect an infrared registration image;

[0079] In S22, the process of collecting feature points is: using the imfindcircles() function in the MATLAB software to solve the coordinates of all circle centers in the visible light registration image and the infrared registration image. In this embodiment, the circular coordinates of the first and ninth circles in the first row, the third and eighth circles in the second row, the fourth circle in the third row, the third and eighth circles in the fifth row, and the first and ninth circles in the sixth row in the two registration images are selected as feature points to form nine pairs of feature points.

[0080] In S23, the process of solving the basic matrix F using the eight-point method is as follows: first, construct a matrix A = [[x1x′1,x1y′1,x1,y1x′1,y1y′1,y1,x′1,y1,1],…,[x9x′9,x9y′9,x9,y9x′9,y9y′9,y9,x′9,y9,1]], then establish a matrix equation AF = 0, substitute nine pairs of characteristic points into the equation, and perform singular value decomposition A = UDV T, then the solution of the basic matrix F is the last column vector of the matrix V, thus completing the solution of the basic matrix F. Among them, the basic matrix F is used to describe the geometric relationship between the two views and is a 3×3 matrix.

[0081] In S24, the process of decomposing the basic matrix F by SVD and then solving the camera extrinsic parameter matrix is:

[0082] Perform SVD singular value decomposition on the basic matrix F = UΣV T , then the translation vector T = ±U1×U2, U1 and U2 are the first and second columns of U, and the rotation matrix det(U) is the determinant of the U matrix, thereby completing the solution of the camera extrinsic matrix and realizing camera registration.

[0083] Furthermore, the elevator control cabinet component data set is obtained, such as Figure 4 ,include:

[0084] Based on the visible light camera, several visible light images of elevator control cabinets are collected to form a data set, and target annotation is performed on several visible light images of elevator control cabinets to obtain data set labels;

[0085] The categories of the data set labels include control circuit boards, microprocessors, contactors, and relays; and the label information of the data set labels includes the category, x coordinate, y coordinate, w width, and h height information of each target.

[0086] Specifically, a visible light camera is used to collect several visible light images of the elevator control cabinet to form a data set image. The first data set image is imported into the X-AnyLabeling software, and the box labeling function is used to label all the elevator control cabinet components in the image. That is, the elevator control cabinet components are selected using boxes, and data is labeled for all data set images, thereby completing the data set label acquisition.

[0087] Among them, the dataset label categories include elevator control cabinet components such as control circuit boards, microprocessors, contactors, and relays. The dataset label information includes five pieces of information for each target: category, x-coordinate, y-coordinate, w width, and h height.

[0088] Furthermore, the target detection network model is trained by a training set and verified by a validation set, and both the training set and the validation set are selected from the elevator control cabinet component data set;

[0089] The target detection network model includes an input layer, a hidden layer and an output layer, wherein the input layer is used to input a visible light image of an elevator control cabinet; the hidden layer is composed of a number of convolutional layers with different convolutional kernel sizes and different numbers of convolutional kernels; and the output layer is used to output the positions of all visible light elevator control cabinet components in the visible light image of the elevator control cabinet.

[0090] Specifically, in this embodiment, the dataset images and their corresponding dataset labels are divided into an elevator control cabinet training set, an elevator control cabinet verification set, and an elevator control cabinet test set according to 8:1:1.

[0091] Furthermore, training the target detection network model includes (such as Figure 5 ):

[0092] S1. Input the elevator control cabinet training set into the input layer to form a tensor feature map, forward propagate the tensor feature map, extract features after passing through the hidden layer, process the extracted features based on the output layer, and output the visible light elevator control cabinet component position;

[0093] S2, inputting the visible light elevator control cabinet component position and the elevator control cabinet test set into the loss function, calculating the loss value, and back-propagating along the gradient direction of the loss function through the loss value to update the model weight parameters;

[0094] S3. Repeat S1-S2 until the average accuracy is greater than the threshold, and obtain the optimal target detection model to be determined, that is, the target detection network model.

[0095] Specifically, the elevator control cabinet training set is input into the input layer to form a tensor feature map, the tensor feature map is forward propagated, the features are extracted after passing through the hidden layer, and finally the output layer processes the extracted features to output the visible light elevator control cabinet component position;

[0096] Input the visible light elevator control cabinet component positions and the elevator control cabinet test set into the loss function and calculate the loss value;

[0097] The target detection model performs backpropagation along the gradient direction of the loss function based on the loss value to update the model weight parameters.

[0098] Repeat the above three steps until the average accuracy is greater than the threshold, and the optimal target detection model is obtained.

[0099] Repeat the model training several times to obtain several undetermined optimal target detection network models, input the elevator control cabinet test set into several undetermined target detection network models, and take the undetermined optimal target detection network model corresponding to the highest average accuracy as the optimal target detection network model.

[0100] Furthermore, the positions of the elevator control cabinet components are mapped to the infrared image of the elevator control cabinet through the camera external parameter matrix, and the temperature of the corresponding elevator control cabinet components is calculated based on the infrared image to achieve temperature matching and real-time monitoring of the elevator control cabinet components, including:

[0101] The visible light elevator control cabinet component position is multiplied by the rotation matrix and added with the translation vector to obtain the infrared elevator control cabinet component position. The average and peak values ​​of the corresponding temperatures of all pixels in each infrared elevator control cabinet component position are solved to complete the elevator control cabinet component temperature matching. The temperature of each elevator control cabinet component is monitored in real time. When the temperature is greater than the set value, an abnormal warning is issued.

[0102] Specifically, the process of mapping the visible light elevator control cabinet component position to the infrared elevator control cabinet component position through the rotation matrix R and the translation vector T includes: multiplying the visible light elevator control cabinet component position with the rotation matrix R and adding the translation vector T to obtain the infrared elevator control cabinet component position.

[0103] The average and peak values ​​of the temperatures corresponding to all pixel points in each infrared elevator control cabinet component position are solved to complete the temperature matching of the elevator control cabinet components.

[0104] The temperature of each elevator control cabinet component is monitored in real time. When it is greater than the set threshold, an abnormal warning is issued, so that the elevator can be repaired in time to ensure the safe operation of the elevator.

[0105] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for monitoring an elevator control cabinet based on the fusion of visible light and infrared images, characterized in that: include: Collecting and optimizing an initial elevator control cabinet image to obtain an optimized elevator control cabinet image, wherein the initial elevator control cabinet image includes an initial elevator control cabinet visible light image and an initial elevator control cabinet infrared image; Align the visible light camera and the infrared camera to obtain the camera extrinsic parameter matrix; Acquire an elevator control cabinet component data set, and build a target detection network model based on the elevator control cabinet component data set; The optimized elevator control cabinet image is input into the target detection network model to obtain the position of the visible light elevator control cabinet components, and the position of the elevator control cabinet components is mapped to the elevator control cabinet infrared image through the camera extrinsic parameter matrix. The temperature of the corresponding elevator control cabinet components is calculated based on the infrared image to achieve elevator control cabinet component temperature matching and real-time monitoring.

2. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 1 is characterized in that: Acquire the initial elevator control cabinet image and optimize it, including: The initial visible light image of the elevator control cabinet and the initial infrared image of the elevator control cabinet are collected respectively, and the initial visible light image of the elevator control cabinet is subjected to distortion correction to obtain an optimized visible light image of the elevator control cabinet. The initial infrared image of the elevator control cabinet is subjected to distortion correction and bilateral filtering to obtain an optimized infrared image of the elevator control cabinet.

3. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 2 is characterized in that: The process of performing the optimization includes: Calibrate the visible light camera and the infrared camera to obtain the visible light camera internal parameter matrix and the infrared camera internal parameter matrix, wherein the visible light camera is used to collect the visible light image of the elevator control cabinet, and the infrared camera is used to collect the infrared image of the elevator control cabinet; Using the visible light camera internal parameter matrix to perform distortion correction on the initial elevator control cabinet visible light image to obtain an elevator control cabinet visible light image; The infrared camera internal parameter matrix is ​​used to perform distortion correction on the initial elevator control cabinet infrared image, the distortion-corrected elevator control cabinet infrared image is output, and bilateral filtering is used to perform noise reduction on the image to obtain the elevator control cabinet infrared image, thereby completing image optimization.

4. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 3 is characterized in that: Calibrating the visible light camera and the infrared camera includes: Collect a number of visible light calibration plate images and infrared calibration plate images respectively, solve all the center coordinates in each of the visible light calibration plate images and the infrared calibration plate images respectively, and use Zhang's calibration method to obtain the visible light camera internal parameter matrix and the infrared camera internal parameter matrix to complete the calibration; When the visible light camera is used to collect the image of the visible light calibration plate, the calibration plate is directly photographed; when the infrared camera is used to collect the image of the infrared calibration plate, the LED light is powered on first, and then the calibration plate is photographed when the heating is stable.

5. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 1 is characterized in that: Align the visible light camera and the infrared camera to obtain the camera extrinsic matrix, including: Collecting a visible light registration image and an infrared registration image respectively, solving the center coordinates of circles in the visible light registration image and the infrared registration image, and selecting a number of center coordinates as feature points; The eight-point method is used to solve the feature points to obtain the camera basic matrix; The basic matrix is ​​decomposed based on SVD to obtain a rotation matrix and a translation vector, thereby completing camera registration; wherein the camera extrinsic parameter matrix includes the rotation matrix and the translation vector.

6. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 1 is characterized in that: Get the elevator control cabinet component data set, including: Based on the visible light camera, several visible light images of elevator control cabinets are collected to form a data set, and target annotation is performed on several visible light images of elevator control cabinets to obtain data set labels; The categories of the data set labels include control circuit boards, microprocessors, contactors, and relays; and the label information of the data set labels includes the category, x coordinate, y coordinate, w width, and h height information of each target.

7. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 1 is characterized in that: The target detection network model is trained by a training set and verified by a verification set, wherein both the training set and the verification set are selected from the elevator control cabinet component data set; The target detection network model includes an input layer, a hidden layer and an output layer, wherein the input layer is used to input a visible light image of an elevator control cabinet; the hidden layer is composed of a plurality of convolutional layers with different convolutional kernel sizes and different numbers of convolutional kernels; and the output layer is used to output the positions of all visible light elevator control cabinet components in the visible light image of the elevator control cabinet.

8. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 7 is characterized in that: The process of training the target detection network model includes: S1. Input the elevator control cabinet training set into the input layer to form a tensor feature map, forward propagate the tensor feature map, extract features after passing through the hidden layer, process the extracted features based on the output layer, and output the visible light elevator control cabinet component position; S2, inputting the visible light elevator control cabinet component position and the elevator control cabinet test set into the loss function, calculating the loss value, and back-propagating along the gradient direction of the loss function through the loss value to update the model weight parameters; S3. Repeat S1-S2 until the average accuracy is greater than the threshold, and obtain the optimal target detection model to be determined, that is, the target detection network model.

9. The elevator control cabinet monitoring method based on visible light and infrared image fusion according to claim 1 is characterized in that: The positions of the elevator control cabinet components are mapped to the infrared image of the elevator control cabinet through the camera external parameter matrix, and the temperature of the corresponding elevator control cabinet components is calculated based on the infrared image to achieve temperature matching and real-time monitoring of the elevator control cabinet components, including: The visible light elevator control cabinet component position is multiplied by the rotation matrix and added with the translation vector to obtain the infrared elevator control cabinet component position, and the average and peak values ​​of the temperatures corresponding to all pixel points in each infrared elevator control cabinet component position are solved to complete the elevator control cabinet component temperature matching, and the temperature of each elevator control cabinet component is monitored in real time. When the temperature is greater than the set value, an abnormal warning is issued.

Images