Elevator fault accident identification method and system based on image identification

Abstract

The invention relates to the technical field of image feature extraction and matching, in particular to an elevator fault accident recognition method and system based on image recognition, and the method comprises the following steps: collecting an elevator image positioning slide rail structure area, extracting an edge line to generate a closed graph, analyzing a door body texture extension mode, calibrating a chain driving area, and tracking a texture moving direction. Door body boundary structure changes are compared, and finally an elevator fault feature matching image recognition result is output. According to the invention, by establishing a linkage mechanism of image boundary extraction, texture direction analysis and trajectory tracking, spatial correlation analysis among a slide rail, a door body and a chain structure is realized, the image recognition precision of door body tiny deformation and chain motion abnormity is enhanced, and potential fault features are extracted based on difference comparison and density comparison. The structure change tracking capability and the time sequence integrity of image expression are improved, the visual judgment capability of complex structure abnormity is enhanced, and the application of image recognition in elevator operation monitoring is expanded.

Description

Technical Field

[0001] This invention relates to the field of image feature extraction and matching technology, and in particular to an elevator malfunction and accident identification method and system based on image recognition. Background Technology

[0002] Image feature extraction and matching technology involves analyzing and processing representative information in images to achieve goals such as image comparison, recognition, and classification. The core of this technology includes image acquisition, preprocessing, key feature point extraction, feature descriptor generation, and feature matching algorithms. Through algorithmic means, it identifies the inherent relationships between elements such as structure, texture, and edges in images, thereby supporting semantic-level understanding and discrimination of image content in different scenarios. It is widely used in various practical fields such as intelligent monitoring, fault detection, facial recognition, and target tracking. Traditional elevator fault identification methods rely on sensors to collect equipment status information or manual inspection records to identify and judge potential door system anomalies, operational interruptions, jams, or other faults that may occur during elevator operation. Common methods include current detection, voltage fluctuation analysis, relay response time monitoring, alarm system trigger judgment, and operation log comparison to determine the fault type and cause. These methods depend on a large number of on-site installed physical sensors and manual intervention for status judgment.

[0003] Existing technologies rely on physical sensors to acquire elevator operating status signals. When faced with structural anomalies, they lack the ability to comprehensively perceive the spatial morphology and detailed evolution of components. Sensors struggle to identify trajectory changes in chain drive parts and door boundary deformation processes, making it difficult to detect early signs of faults such as door offset and slide rail wear. The identification process is highly dependent on single-point data and cannot form a global discrimination model for structurally related areas. It lacks semantic understanding of fine-grained structural states in image space. In complex scenarios or under multi-factor interference, fault identification errors are prone to occur, and it cannot effectively support the tracking and judgment of dynamic structural evolution trends in elevator door systems. This results in unclear fault source location, inability to trace the evolution process, and difficulty in closing the loop of response mechanisms. Summary of the Invention

[0004] To achieve the above objectives, the present invention adopts the following technical solution: an elevator fault accident identification method based on image recognition, comprising the following steps: S1: Collect image data during elevator operation, locate the visual region in the image corresponding to the elevator car slide rail connection structure, segment the outer edge line segment of the visual region, and output the car slide rail boundary contour image content. S2: Call the content of the car slide rail boundary contour image, identify the projection area of ​​the elevator door in the image, analyze the directional extension of the door texture in the projection area, extract the strip distribution pattern, and output the texture distribution image content of the door area; S3: Call the texture distribution image content of the door area, mark the chain driving area within the door boundary, track the movement direction of the chain texture in consecutive image frames, and output the chain motion trajectory image path; S4: Call the chain motion trajectory image path, compare the door boundary within the path coverage area in multiple frames, mark the structural changes at different time points, construct visual comparison graphics, and output the difference image content of the door deformation parts; S5: Call the difference image content of the deformed part of the door, the image path of the chain movement trajectory, and the image content of the boundary contour of the car slide rail, compare the edge overlap, and output the elevator fault feature matching image recognition result.

[0005] As a further aspect of the present invention, the car slide rail boundary contour image content includes structural boundary lines, contour closed areas, and slide rail connection features; the door body area texture distribution image content includes texture extension direction, strip arrangement pattern, and surface distribution features; the chain motion trajectory image path includes motion direction lines, trajectory continuity graphics, and chain position sequence; the door body deformation part difference image content includes boundary offset area, structural deformation position, and time difference layer; and the elevator fault feature matching image recognition result includes structural interlacing distribution, abnormal image overlap, and fault feature marking.

[0006] As a further aspect of the present invention, the tracking of the movement direction of the chain texture in continuous image frames refers to verifying the movement direction and trajectory of the chain in the time series by identifying and matching the temporal changes of the chain surface texture features in continuously captured image frames.

[0007] As a further aspect of the present invention, the directional extension method of the door texture refers to analyzing the arrangement trend and directional changes of the texture on the surface of the elevator door in the image, and identifying the dominant extension direction and distribution pattern in space.

[0008] As a further aspect of the present invention, the specific steps of S1 are as follows: S101: Acquire image data frames during elevator operation, perform spatial domain enhancement based on image grayscale distribution characteristics, extract grayscale abnormal regions corresponding to the slide rail connection structure in the image, filter and cluster pixels with grayscale gradient values ​​higher than the preset edge response threshold, and obtain the initial pixel set of the visual region of the slide rail connection structure. S102: Call the initial pixel set of the visual region of the sliding rail connection structure, construct a connection map according to the two-dimensional coordinate relationship of the pixel points, perform morphological gradient processing on the boundary of the connection path, filter the boundary pixel point group with stable gradient direction and amplitude higher than the edge reference value, and obtain the outer edge line segment sequence of the structure region. S103: Call the sequence of line segments on the outer edge of the structure region, determine the connection relationship based on the start and end coordinates of the line segments, filter the combination of line segments that meet the closed loop judgment benchmark value to form a closed region, and generate the boundary contour image content of the car slide rail.

[0009] As a further aspect of the present invention, the specific steps of S2 are as follows: S201: Call the content of the car slide rail boundary contour image, filter the set of regions with brightness changes and closed boundaries in the adjacent regions according to the structural boundary position in the image coordinate space, and perform geometric alignment verification on the edge of the region shape and the slide rail contour. After filtering out the regions that do not meet the alignment conditions, obtain the door projection area image fragment. S202: Call the image segment of the projection area of ​​the door body, calculate the concentration of the gradient direction angle of adjacent pixels based on the direction of image gray value change, filter the pixel group with the direction concentration higher than the preset texture direction judgment threshold, and generate a texture direction extension pattern structure. S203: Invoke the texture direction extension pattern structure, and perform directional clustering on the positional relationship in the image space according to the direction vector and distribution density of the texture strips in the guide area to obtain the texture distribution image content of the door area.

[0010] As a further aspect of the present invention, the specific steps of S3 are as follows: S301: Call the texture distribution image content of the door area, retrieve the texture area enclosed by the door boundary in the image coordinate space, and filter the sub-regions with texture density higher than the preset chain texture filtering benchmark value based on the texture strip density and direction consistency parameters in the region, and mark the corresponding position in the image to obtain the chain drive area position index set; S302: Call the chain drive region position index set to obtain the image block of the corresponding coordinate region in the continuous image frame, and generate the chain texture direction change sequence according to the angle change trend of the texture direction in the image block and the coordinate displacement of the texture feature point between the continuous frames. S303: Call the chain texture direction change sequence, perform path combination operation based on the continuity of texture position offset vectors between adjacent frames, and perform topological verification on the connection relationship of all path segments. After filtering out non-continuous segments, obtain the chain motion trajectory image path.

[0011] As a further aspect of the present invention, the specific steps of S4 are as follows: S401: Call the image path of the chain motion trajectory, extract the image coordinate block within the trajectory coverage area, obtain the set of door boundary pixels in consecutive image frames, perform inter-frame comparison based on the gradient direction of boundary pixels under the same spatial position, and mark the pixel positions where the angle difference is greater than the preset door boundary difference threshold to obtain the boundary structure change mark index set. S402: Call the boundary structure change marker index set, count the time points when the marker index appears in the image frame sequence, extract the temporal information of boundary changes according to the time distribution law, select the image frames whose difference value first exceeds the boundary difference judgment benchmark value as the differential frames, and generate the door boundary difference time node sequence. S403: Call the time node sequence of the door boundary difference, extract the boundary image block of the marked index coverage area in the corresponding image frame, perform pixel alignment and gray level difference operation on the frame images before and after the time node, and construct a visual mapping map based on the gray level difference to obtain the difference image content of the door deformation part.

[0012] As a further aspect of the present invention, the specific steps of S5 are as follows: S501: Call the difference image content of the deformed part of the door, the image path of the chain movement trajectory and the image content of the boundary contour of the car slide rail, perform pixel alignment operation on the three types of images in a unified image coordinate system, and overlay the image layers according to the layer identifier to obtain the structural information overlay image matrix. S502: Call the structural information to overlay the image matrix, divide the image region into equally spaced grid blocks, extract the texture edge values, direction vectors and number of overlapping markers in each grid block, calculate the proportion of the number of overlapping structures to the total number of boundary pixels, and compare it with the feature overlap index to obtain the image region edge overlap distribution value. S503: Call the edge overlap distribution value of the image region, and according to the distribution difference of density values ​​in the image space, search for regions with density higher than the preset fault identification threshold and mark the coordinates of the center point of the region. Combine the texture pattern of the corresponding region image block to establish a recognition feature set and obtain the elevator fault feature matching image recognition result.

[0013] An elevator malfunction and accident identification system based on image recognition includes: The slide rail boundary extraction module is used to achieve S1: collecting image data during elevator operation, locating the visual region in the image corresponding to the elevator car slide rail connection structure, segmenting the outer edge line segment of the visual region, and outputting the slide rail boundary contour image content; The door texture analysis module is used to implement S2: call the content of the car slide rail boundary contour image, identify the projection area of ​​the elevator door in the image, analyze the directional extension mode of the door texture in the projection area, extract the strip distribution pattern, and output the texture distribution image content of the door area; The chain trajectory tracking module is used to implement S3: call the texture distribution image content of the door area, mark the chain driving area within the door boundary, track the movement direction of the chain texture in continuous image frames, and output the chain motion trajectory image path; The door deformation detection module is used to implement S4: call the chain motion trajectory image path, compare the door boundary within the path coverage area in multiple frames, mark the structural changes at different time points, construct visual comparison graphics, and output the difference image content of the door deformation part; The fault feature recognition module is used to implement S5: call the difference image content of the door body deformation part, the image path of the chain movement trajectory and the image content of the car slide rail boundary contour, compare the edge overlap, and output the elevator fault feature matching image recognition result.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by establishing a linkage mechanism of image boundary extraction, texture direction analysis and trajectory tracking, the spatial correlation analysis between the slide rail, door body and chain structure is realized, the image recognition accuracy of minor deformation of the door body and abnormal chain movement is enhanced, potential fault features are extracted based on difference comparison and density comparison, the ability to track structural changes and the temporal integrity of image expression are improved, the visualization and discrimination ability of complex structural anomalies is strengthened, and the application depth of image recognition in elevator operation monitoring is expanded. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a detailed schematic diagram of S1 of the present invention; Figure 3 This is a detailed schematic diagram of S2 of the present invention; Figure 4 This is a detailed schematic diagram of S3 of the present invention; Figure 5 This is a detailed schematic diagram of S4 of the present invention; Figure 6This is a detailed schematic diagram of S5 of the present invention; Figure 7 This is a system module diagram of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] Please see Figure 1 This invention provides an elevator malfunction accident identification method based on image recognition, comprising the following steps: S1: By collecting image data of the elevator during operation, locate the visual area in the image corresponding to the connection structure of the elevator car slide rail, segment the outer edge line segments of the structural area, and combine the line segments to form a continuous closed image area, and output the boundary contour image content of the car slide rail. S2: Call the car slide rail boundary contour image content, verify the projection area of ​​the elevator door in the image, analyze the direction extension of the door texture in the image, extract the distribution pattern of the texture stripes, and output the texture distribution image content of the door area. S3: Call the texture distribution image content of the gate area, calibrate the chain driving area within the gate boundary, track the movement direction of the chain texture in consecutive image frames and combine them to form a path graphic, and output the chain motion trajectory image path; S4: Call the chain motion trajectory image path, compare multiple frames of images of the door boundary in the trajectory coverage area, calibrate the structural changes of the boundary at different time points, construct visual comparison graphics, and output the difference image content of the door deformation parts. S5: Call the difference image content of the door deformation part, the image path of the chain movement trajectory and the image content of the car slide rail boundary contour, and superimpose the three types of image content to form a fused image. Compare the edge overlap of the image regions in the fused image and output the elevator fault feature matching image recognition result.

[0023] The car slide rail boundary contour image includes structural boundary lines, contour closed areas, and slide rail connection features. The door area texture distribution image includes texture extension direction, strip arrangement pattern, and surface distribution features. The chain movement trajectory image path includes movement direction lines, trajectory continuity graphics, and chain position sequence. The door deformation part difference image includes boundary offset area, structural deformation position, and time difference layer. The elevator fault feature matching image recognition results include structural interlacing distribution, abnormal image overlap, and fault feature marking.

[0024] Please see Figure 2 The specific steps of S1 are as follows: S101: Acquire image data frames during elevator operation, perform spatial domain enhancement based on image grayscale distribution characteristics, extract grayscale abnormal regions corresponding to the slide rail connection structure in the image, filter and cluster pixels with grayscale gradient values ​​higher than the preset edge response threshold, and obtain the initial pixel set of the visual region of the slide rail connection structure. Acquiring image data frames during elevator operation refers to the continuous image capture by a fixed camera installed on the top or side of the elevator car during its vertical movement. This camera should be configured to provide sufficient spatial coverage at the elevator's operating speed. For example, when the elevator is moving at 1.5 meters per second, capturing 30 frames per second can achieve a vertical resolution of approximately 5 centimeters. After image capture, grayscale processing is first performed. This involves single-channel conversion of the color image according to the red, green, and blue channel weights of each pixel, ensuring that each pixel reflects its brightness information. The converted grayscale image then undergoes further noise reduction processing. This process uses a fixed-size sliding window to traverse the image pixel by pixel, selecting the median grayscale value within the window to replace the current pixel value, smoothing out isolated high-noise points. Next, the overall grayscale distribution characteristics of the image are enhanced. This involves analyzing the concentrated areas of grayscale value distribution in the image and extending the dynamic range of the grayscale by linear stretching, making the difference between bright and dark areas more pronounced. This operation is based on the image's... The minimum and maximum gray values ​​are mapped between intervals, which enhances the image and makes the boundaries between light and dark areas more prominent, which helps in the subsequent identification of the sliding rail connection structure. Next, the location of abnormal gray value distribution in the image is analyzed by comparing the average difference of the gray value of the pixel with that of its neighboring pixels. When the difference exceeds a specified threshold, the pixel is marked as an anomaly. The threshold is set to 15 based on the overall background gray value noise range of the image. Then, all regions marked as anomalies are further judged by calculating the degree of gray value change in the horizontal and vertical directions at each pixel position and merging the gradient value of the point. When the gradient value exceeds the set response threshold of 40, the point is recorded as a potential edge point. Then, these pixels that meet the conditions are aggregated into an initial pixel cluster according to their spatial location. The judgment method is to search for multiple other pixels that meet the conditions in a 3×3 area centered on each pixel. If there are 5 or more pixels that meet the conditions, they are clustered into the same region. This process is repeated to traverse the entire image to obtain the initial pixel set of the visual region of the sliding rail connection structure.

[0025] S102: Call the slider to connect the initial pixel set of the visual region of the structure, construct the connection graph according to the two-dimensional coordinate relationship of the pixel points, perform morphological gradient processing on the boundary of the connection path, filter the boundary pixel group with stable gradient direction and amplitude higher than the edge reference value, and obtain the sequence of outer edge line segments of the structure region. Further analysis of the boundaries and shapes of the extracted gray-level anomaly regions in the image is conducted to construct complete structural shape information. First, the spatial position of each pixel is determined based on its row and column coordinates in the image. Then, a search is performed within its eight adjacent pixels to determine if pixels belonging to the same cluster exist. If so, a connection is established between the two pixels. All pixels satisfying this connection rule are combined into a connection graph, which shows the connection paths of pixels in the slide rail structure region. Subsequently, morphological processing operations are performed on the pixels at the edges of this connection graph. This involves performing a dilation operation on the image, expanding the boundary outward by one pixel unit, and then subtracting the original image from the expanded image. This process extracts the edge contours, which are then analyzed for grayscale change direction. For each boundary pixel, the grayscale change direction of its adjacent pixels in the image is calculated. If the grayscale change direction of a point in its front and back directions remains within ±10 degrees, and its grayscale change amplitude exceeds the edge reference value (i.e., the value that is in the top 10% after sorting the grayscale change amplitude of all edge pixels in the image), this value of 120 is taken as the judgment criterion. Only boundary pixels that simultaneously satisfy both directional stability and large change amplitude are retained. Finally, these highly stable and high-amplitude boundary pixels are concatenated to form a series of edge line segment sequences outside the structural region, which are used for the next step of boundary closure judgment and contour generation processing.

[0026] S103: Call the sequence of line segments on the outer edge of the structural region, determine the connection relationship based on the start and end coordinates of the line segments, filter the combination of line segments that meet the closed loop judgment benchmark value to form a closed region, and generate the car slide rail boundary contour image content. After extracting the outline of the sliding rail structure, the spatial arrangement and connection relationship of these line segments are further judged and combined to construct a closed boundary. First, the start and end coordinates of each edge line segment are extracted. By calculating the Euclidean distance between the start and end points of any two line segments, if the distance is less than 5% of the width of the sliding rail image, i.e., about 10 pixels, it is determined that the two line segments are connected. Then, these line segments are combined into a line segment chain to form a possible closed region. To determine whether the region is closed, the start and end points of the line segment chain need to be recalculated. If the distance between the two points is less than 2 pixels and the combination contains at least four edge line segments, the line segment chain is considered to form a closed figure. All closed line segment combinations that meet this condition are retained to generate the region boundary image. Then, a mask filling operation is performed on these closed regions on the original image, that is, all pixels in the closed region are marked as pixels in the structural region, and their corresponding image blocks are extracted as the final output result, forming the complete car sliding rail boundary outline image content.

[0027] Please see Figure 3 The specific steps of S2 are as follows: S201: Call the content of the car slide rail boundary contour image, filter the set of regions with brightness changes and closed boundaries in the adjacent regions according to the structural boundary position in the image coordinate space, and perform geometric alignment verification on the edge of the region shape and the slide rail contour. After filtering out the regions that do not meet the alignment conditions, obtain the image fragment of the door projection area. First, the location range of the structural boundary is determined in the image. By traversing the edge pixel coordinates of the slider contour image, a rectangular box region surrounding the boundary is constructed, and a certain number of pixels are extended outward as the analysis range of adjacent regions. For example, the neighborhood range is defined as 20 pixels outward from the slider boundary. Then, the brightness changes of each pixel region within this range are scanned. For each candidate region, the difference between the maximum and minimum gray values ​​of the pixels within the region is calculated. When this difference exceeds the set brightness change threshold of 60, the region is recorded as a brightness change region. Next, it is further determined whether the brightness region constitutes a closed boundary by detecting whether all edge pixels in the region can form a closed contour line. If the distance between the starting point and the ending point coordinates of the contour line is small... If a region has at least 3 pixels and at least 4 line segments, it is considered a closed region. Then, geometric alignment is checked on all closed regions. This involves comparing the boundary line segments of each closed region with the edges of the original slide rail contour image segment by segment. If the angle difference between the corresponding line segments is less than 15 degrees, and the center point of the candidate region is approximately coincident with the center point of the slide rail contour region in the image coordinate system (e.g., the relative distance is no more than 10 pixels, and the spatial structures of the two are comparable in the image viewpoint), then the region is considered to have consistent geometric features with the slide rail structure and meets the geometric alignment condition. Otherwise, the region is discarded. Finally, all regions that meet the geometric alignment condition are uniformly marked as door projection regions, and their fragment content in the image is extracted as output image fragments.

[0028] S202: Call the image fragment of the projection area of ​​the door, calculate the concentration of the gradient direction angle of adjacent pixels based on the direction of change of the gray value of the image, filter the pixel group with the direction concentration higher than the preset texture direction judgment threshold, and generate the texture direction extension pattern structure. The process involves analyzing the image texture direction within the identified door area. First, the grayscale values ​​of all pixels in the image are extracted. Then, grayscale direction analysis is performed on the window formed by each pixel and its neighboring pixels, calculating the difference in grayscale changes in the horizontal and vertical directions. Next, the gradient direction angle of the current pixel is calculated based on the direction of the difference. The gradient direction angles of all pixels are recorded to form an angle distribution map. Then, the concentration of all gradient direction angles within a 5×5 window around each pixel is calculated by statistically analyzing the differences between the direction angles within the window and calculating their standard deviation. The lower the standard deviation, the more concentrated the direction. A texture direction judgment threshold is then set. The standard deviation of the gradient direction is less than 25 degrees, indicating that the region has significant directionality. The region where the pixel is located is marked as having a concentrated texture direction. Subsequently, all pixels that meet the direction concentration condition are combined into a texture structure region, and a boundary completion operation is performed on it. That is, adjacent concentrated pixels are extended and connected through a region growing method. Finally, an extended pattern structure containing various texture directions is generated, and its region position, direction angle range, boundary shape and other information are labeled and output.

[0029] S203: Call the texture direction extension pattern structure, and perform directional clustering on the positional relationship in the image space according to the direction vector and distribution density of the texture strips in the guide area to obtain the texture distribution image content of the door area; This process involves performing directional feature-based clustering on the generated texture structure regions to extract complete texture distribution information for the door region. First, the direction vector information of each texture strip is obtained. This direction vector is determined by the average gradient direction of continuous pixels within the strip. Then, the starting and ending coordinates of each texture strip in the image coordinate space are calculated, and its position coordinates in the image are extracted with the midpoint of the strip as the center. Subsequently, the density distribution of each strip in the image is statistically analyzed. The image is divided into fixed-size grid cells in the image space, for example, the entire image is divided into 40×40 pixel cells. The number of texture strips in each cell is counted, and their average direction vector is calculated. Next, directional clustering is performed on the average direction vectors of all cells. The clustering method is to group grid cells with a directional difference of less than 20 degrees and a strip density of more than 3 strips into one class. By repeatedly traversing all texture structure regions in the image, regions with similar directions and concentrated distributions are grouped together to form texture sub-regions. Then, contour closure and boundary correction are performed on each sub-region to extract continuously distributed door texture regions. Finally, the corresponding door region texture distribution image content is formed in the image.

[0030] Please see Figure 4 The specific steps of S3 are as follows: S301: Call the texture distribution image content of the gate area, retrieve the texture area enclosed by the gate boundary in the image coordinate space, filter the sub-regions with texture density higher than the preset chain texture filtering benchmark value based on the texture strip density and direction consistency parameters in the region, mark the corresponding position in the image, and obtain the chain drive area position index set; Based on the extracted image regions with clear texture direction and density characteristics, the texture stripes within their boundary ranges are statistically analyzed. First, a corresponding region is selected in the image coordinate space using the gate boundary as a reference. A closed polygon is formed by recording the boundary vertex positions. Then, all texture stripe pixels within this polygon are traversed based on pixel coordinates. Each local region is divided into 10×10 pixel units. The number of texture stripe pixels in each unit is counted as the texture density value. Simultaneously, the direction vectors of all texture stripes in that cell are extracted, and the direction consistency parameter is calculated based on their angle differences. The direction consistency parameter is obtained by calculating the standard deviation of the texture direction within each region. When this standard deviation is below 20 degrees, and the texture stripes within that region... When the density value is higher than the preset benchmark value, the sub-region is determined to be a texture concentration region. The chain texture screening benchmark value is set according to the average texture density of the entire image as the 70th percentile of the density of all 10×10 cells in the image. For example, in the sample image, the density corresponding to this quantile value is 30. Then, the region with a density exceeding 30 is marked, and the sub-regions with an orientation consistency parameter greater than 20 degrees are filtered out. Only the regions that meet the dual conditions are retained as chain candidate regions. Then, the pixel coordinates of the upper left and lower right corners of these regions are recorded in the image coordinates to form a rectangular index data structure. The positions of these regions are marked on the image by using a bounding box. Finally, the position index data set of all sub-regions that meet the dual conditions of density and orientation is output as the chain driving region position index set.

[0031] S302: Call the chain-driven region position index set to obtain the image patch of the corresponding coordinate region in the continuous image frame, and generate the chain texture direction change sequence according to the angle change trend of the texture direction in the image patch and the coordinate displacement of the texture feature point between the continuous frames. Using the sub-region location information obtained in the previous step, image content tracking is performed on the same coordinate region in a continuously acquired image frame sequence. First, based on the coordinate frame position of each region in the index set, the corresponding image block is extracted in each frame. Then, the main texture direction inside the image block is calculated in each frame. This main texture direction is obtained by statistically analyzing the directions of all texture stripes in the image block and calculating their average angle. If there are multiple direction stripes in the image block, the main direction angle value is obtained by weighting each texture direction according to its pixel count. Subsequently, the main texture directions extracted from the same index region in adjacent image frames are processed. The difference between two frames is calculated, that is, the difference between the principal direction angles of the corresponding regions in frame t and frame t+1 is used as the change in texture direction between frames. The angle difference is recorded as a sequence in chronological order. This sequence reflects the change trend of texture direction in the chain-driven region in consecutive frames. For example, if the principal direction angles of the corresponding regions in frames 1, 2, and 3 are 32 degrees, 35 degrees, and 41 degrees, then the frame difference sequence is 3 degrees and 6 degrees. This sequence is used to determine the direction and continuity of chain movement in subsequent processing. Finally, this process is completed in all frames and arranged in chronological order, and the output is the chain texture direction change sequence.

[0032] S303: Call the chain texture direction change sequence, perform path combination operation based on the continuity of texture position offset vectors between adjacent frames, and perform topological verification on the connection relationship of all path segments. After filtering out non-continuous segments, the chain motion trajectory image path is obtained. Based on the inter-frame angle change information obtained in the previous stage, the continuous texture direction offset vector on the time axis is analyzed, and the spatial trajectory of the chain motion path is deduced accordingly. First, the direction change sequence of each index region is traversed, and the fluctuation amplitude of the direction change and the consistency of the direction offset in three or more consecutive frames are calculated. If the change of the direction offset is kept within ±15 degrees and the change value is continuously increasing or decreasing, it is determined to be a segment of the continuous texture offset path. Then, all continuous segments that meet the conditions are merged into a complete path. The connection relationship between paths is determined by comparing the coordinate distance between the end position and the start position of adjacent path segments. If the distance is less than 50% of the width of the index region, it is considered a connectable path segment. Otherwise, the path is determined to be a non-continuous segment and removed from the path set. Further, a topological relationship graph of the path is constructed, and the connection between each path segment is bidirectionally verified, that is, to ensure that the end point of the path and the start point of the next path meet the connection conditions in both position and direction. If a path segment cannot form a complete topological connection with other paths, it is considered an invalid path and is excluded. Finally, the set of all paths that meet the continuity and topological closure conditions constitutes a complete chain motion trajectory, and the output is the chain motion trajectory image path.

[0033] Please see Figure 5 The specific steps of S4 are as follows: S401: Call the chain motion trajectory image path, extract the image coordinate blocks within the trajectory coverage area, obtain the set of gate boundary pixels in consecutive image frames, perform inter-frame comparison based on the gradient direction of boundary pixels at the same spatial position, and mark the pixel positions where the angle difference is greater than the preset gate boundary difference threshold to obtain the boundary structure change mark index set. The image coordinate region covered by the path traversed by the chain movement is extracted based on the image path. Specifically, the pixel coordinates recorded in each frame of the trajectory image path are traversed, and a minimum bounding rectangle containing the coordinates of each trajectory path segment is constructed. This rectangle is used as the image block extraction window, and image data within this region is acquired in each frame. Then, boundary pixel extraction is performed on the known gate boundary positions within this region. This involves identifying all edge pixels with large grayscale variations in the image and confirming that they are within the spatial range of the gate boundary. The image coordinates of these boundary pixels and their corresponding gradient direction angles are recorded. Then, the boundary pixels under the same image coordinates are extracted in consecutive frames. The gradient direction comparison of each pixel is performed by calculating the difference between the direction angle of the pixel in the previous frame and the direction angle of the pixel at the same position in the current frame. When the difference is greater than the gate boundary difference threshold, the pixel is marked as a changed pixel. The difference threshold is set to 15 degrees based on the device imaging error and the allowable jitter range of the gate structure. If a pixel has a direction of 65 degrees in frame n and changes to 90 degrees in frame n+1, the angle difference is 25 degrees, which is greater than 15 degrees and meets the marking condition. This judgment process is performed frame by frame, and the comparison and marking are repeated for each pair of adjacent frames. Finally, the coordinates of all pixel positions that meet the condition of direction angle change exceeding the threshold are summarized, and a boundary structure change mark index set is generated.

[0034] S402: Call the boundary structure change marker index set, count the time points when the marker index appears in the image frame sequence, extract the temporal information of boundary changes according to the time distribution pattern, select the image frames whose difference value first exceeds the boundary difference judgment benchmark value as the differential frames, and generate the door boundary difference time node sequence. The time points of change for each marked location in the image sequence are statistically analyzed. Specifically, all marker indices are iterated one by one, and the frame number in the image frame sequence where the pixel is first marked as a change point is recorded. By organizing and statistically analyzing the first change frame numbers of all change points, a time-pixel relationship mapping table is constructed. Based on this, the temporal distribution pattern of the change of each pixel position is analyzed, that is, to determine in which frame each marker point is identified as a structural change point, and whether the marker continues to appear in subsequent frames. Then, based on this statistical information, the temporal features of boundary changes are extracted, and the frames where the first significant change occurs are identified. As the target analysis point, the judgment criterion is that the number of marked pixels in the frame exceeds 10% of the total number of boundary pixels of the whole frame. For example, if the total number of boundary pixels of a frame is 2000, if the number of marked pixels in the frame is greater than 200, the frame is judged as a differentiated frame. This judgment threshold is used as the boundary difference judgment benchmark value. When setting this benchmark value, the pixel change range of the gate boundary under normal operation is referenced. It is usually set to the range of 10% to 15% of the total boundary pixels. Finally, all image frame numbers that reach or exceed the change benchmark value for the first time are selected and arranged in chronological order to form a gate boundary difference time node sequence.

[0035] S403: Call the time node sequence of the door boundary difference, extract the boundary image block of the marked index covered area in the corresponding image frame, perform pixel alignment and gray level difference operation on the frame images before and after the time node, and construct a visual mapping map based on the gray level difference to obtain the difference image content of the door deformation part. The image content of the changed region in the image frame corresponding to each difference time point is extracted. First, the smallest bounding rectangle region of all changed pixels in the frame is obtained according to the tag index set. The boundary image blocks within this region are extracted. Then, pixel alignment is performed between the difference frame and the previous frame. The method is to align the region center point and adjust the overlap of the boundary image blocks with pixel-level precision so that pixels in the same spatial position can be matched one by one. After alignment, grayscale difference calculation is performed. The grayscale difference of each pair of corresponding pixels is calculated and the change magnitude of the point is represented by the absolute value. All pixel differences are recorded and uniformly mapped to a grayscale view. This mapping uses grayscale levels from black to white to represent the degree of grayscale change of pixels. Pixels with a grayscale difference of less than 10 are set to black (no significant change), pixels with a grayscale difference between 10 and 50 are set to medium gray, and pixels with a grayscale difference of more than 50 are set to white (significant change). Finally, this mapping shows the grayscale difference of all changed regions and is output as an image to reflect the difference image content of the deformed parts of the door.

[0036] Please see Figure 6 The specific steps of S5 are as follows: S501: Call the difference image content of the door deformation part, the image path of the chain movement trajectory and the image content of the car slide rail boundary contour, perform pixel alignment operation on the three types of images in a unified image coordinate system, and overlay the image layers according to the layer identifier to obtain the structural information overlay image matrix. First, the data from three images are read separately. The original resolution, starting coordinates, and coordinate system units of each image are compared. Based on the actual acquisition equipment settings (e.g., all images are 1920×1080 resolution), the coordinate axes of the original images are uniformly converted to the standard image pixel coordinate system, with the top left corner as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis. Then, pixel alignment is performed on each of the three images. The alignment method involves extracting reference coordinates for key structural points in the images. Typically, the area where the edge corner of the slide rail boundary image coincides with the starting point in the chain trajectory image is selected as the anchor point. The anchor point offset is calculated, and the images are subjected to overall translation and rotation transformations to ensure that all images... The structural content completely overlaps in the same spatial location. After spatial matching is completed, the three images are layered and superimposed according to layer logic. The slide rail outline is used as the bottom layer, the chain trajectory image is used as the middle layer, and the door difference image is used as the top layer. Each layer is assigned a unique layer identifier, such as Layer1, Layer2, and Layer3. A separate channel is created for each layer in the image matrix. The pixel values ​​of each layer image are superimposed using a pixel index method. The characteristics of each layer are distinguished by setting the transparency or brightness at the overlapping pixel points, so as to achieve complete preservation of data between layers. Finally, a structural information superimposed image matrix with slide rail, chain, and deformation information is formed.

[0037] S502: Call the structural information to overlay the image matrix, divide the image region into equally spaced grid blocks, extract the texture edge values, direction vectors and number of overlapping markers in each grid block, calculate the proportion of the number of overlapping structures to the total number of boundary pixels, and compare it with the feature overlap index to obtain the image region edge overlap distribution value. First, the image space is regularly divided according to the actual resolution of the image. The entire image is divided into multiple uniformly sized grid blocks according to a fixed pixel spacing. For example, in a common 1920×1080 pixel image, the horizontal and vertical directions are divided into 40-pixel intervals, resulting in multiple sub-regions with clear row and column distributions. Each sub-region is processed as an independent statistical unit in subsequent processing. During the traversal, the pixels in each grid block are read sequentially, and the pixels are recorded according to their affiliation in different image layers. For obtaining texture edge values, it is necessary to determine pixel by pixel whether the point shows a significant grayscale change in the slider layer or chain layer. The determination method is to view the image... The grayscale values ​​of each pixel are compared with those of its neighboring pixels. If the maximum grayscale difference exceeds a preset edge detection threshold, the pixel is recorded as an edge pixel and included in the edge statistics of the current grid block. Simultaneously, the texture direction of each pixel identified as an edge is calculated. The texture direction is obtained by analyzing the grayscale variation trend of the pixel in the horizontal and vertical directions, and its direction angle is uniformly recorded within a standard angle range. In the slider layer, since the slider is a vertical guiding structure, its texture and edge directions are usually concentrated in a near-vertical angle range. Therefore, during statistics, a high concentration of such pixel directions can be observed. In contrast, the texture direction in the chain layer, due to the structural characteristics of the plate chain or roller chain, usually exhibits periodic horizontal or inclined patterns. The two layers have significant differences in directional characteristics. Based on this, a position alignment judgment is performed on pixels from different layers within the same grid block. This involves checking whether there are pixels with completely identical image coordinates that are simultaneously identified as edge pixels in both the slide rail layer and the chain layer. If so, the texture direction difference of the pixel in the two layers is further compared. The direction difference is obtained by directly calculating the absolute difference between two direction angles. Only when this difference is less than the set allowable range for direction angles is the next judgment allowed. Since there is a significant gap between the slide rail and the gantry chain in physical space during normal operation, they should not have a large overlap in the image plane. Therefore, the spatial position of the area to which such pixels belong also needs to be corrected. The verification process involves comparing the center position offsets of the slide rail region and the chain region in the image. If the center offset distance between the two is within a small range, the pixel is recorded as a valid overlapping marker; otherwise, it is directly discarded. After completing the above pixel-by-pixel judgment, the ratio of the number of overlapping markers accumulated in each grid block to the total number of edge pixels in the grid block is calculated to obtain the overlapping structure density value of the grid block. This density value is then compared with the structural overlapping density benchmark value obtained in advance based on a large number of normal operation samples. If the density value exceeds the benchmark value, the spatial position and density value of the corresponding grid block are recorded. Finally, the spatial distribution result of the structural overlapping density of each region is formed within the entire image.

[0038] S503: Call the edge overlap distribution value of the image region, and according to the distribution difference of density values ​​in the image space, search for regions with density higher than the preset fault identification threshold and mark the coordinates of the center point of the region. Combine the texture pattern of the corresponding region image block to establish a recognition feature set and obtain the elevator fault feature matching image recognition result. First, all grid block density values ​​are screened, and areas with density values ​​higher than the fault identification threshold are extracted. The threshold is set with a 10% offset from the median value of the staggered density of the corresponding area in the historical elevator fault data sample as the judgment standard. For example, if the statistical results show that the density of the fault area is concentrated above 0.25, then the threshold is set to 0.25. All grid areas with density values ​​≥ 0.25 are screened and their spatial center point coordinates are marked. The grid center point position is obtained by extracting the coordinates of the upper left corner of each grid block and adding half a grid spacing. At the same time, the texture pattern of the image area corresponding to the grid block is extracted. The texture pattern includes information such as the main direction vector, edge distribution pattern, and staggered point spacing. A pattern vector containing direction, density, and structural shape is constructed. These features are integrated to form a feature set, which is compared with the preset fault template feature set item by item. If there are matching items with a main direction angle offset of less than 10 degrees, an edge number error of less than 15%, and a staggered distribution overlap of more than 80%, then the area is judged as a fault area, and the output is the elevator fault feature matching image recognition result.

[0039] Please see Figure 7 An elevator malfunction and accident identification system based on image recognition includes: The rail boundary extraction module is used to achieve S1: collecting image data during elevator operation, locating the visual region in the image corresponding to the elevator car rail connection structure, segmenting the outer edge line segment of the visual region, and outputting the car rail boundary contour image content; The door texture analysis module is used to implement S2: call the content of the car slide rail boundary contour image, identify the projection area of ​​the elevator door in the image, analyze the directional extension of the door texture in the projection area, extract the strip distribution pattern, and output the texture distribution image content of the door area; The chain trajectory tracking module is used to implement S3: call the texture distribution image content of the door area, mark the chain driving area within the door boundary, track the movement direction of the chain texture in continuous image frames, and output the chain motion trajectory image path; The door deformation detection module is used to implement S4: call the chain motion trajectory image path, compare the door boundary within the path coverage area in multiple frames, mark the structural changes at different time points, construct visual comparison graphics, and output the difference image content of the door deformation parts; The fault feature recognition module is used to implement S5: call the difference image content of the door body deformation part, the chain movement trajectory image path and the car slide rail boundary contour image content, compare the edge overlap, and output the elevator fault feature matching image recognition result.

[0040] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for elevator malfunction and accident identification based on image recognition, characterized in that, Includes the following steps: S1: Collect image data during elevator operation, locate the visual region in the image corresponding to the elevator car slide rail connection structure, segment the outer edge line segment of the visual region, and output the car slide rail boundary contour image content. S2: Call the content of the car slide rail boundary contour image, identify the projection area of ​​the elevator door in the image, analyze the directional extension of the door texture in the projection area, extract the strip distribution pattern, and output the texture distribution image content of the door area; S3: Call the texture distribution image content of the door area, mark the chain driving area within the door boundary, track the movement direction of the chain texture in consecutive image frames, and output the chain motion trajectory image path; S4: Call the chain motion trajectory image path, compare the door boundary within the path coverage area in multiple frames, mark the structural changes at different time points, construct visual comparison graphics, and output the difference image content of the door deformation parts; S5: Call the difference image content of the deformed part of the door, the image path of the chain movement trajectory, and the image content of the boundary contour of the car slide rail, compare the edge overlap, and output the elevator fault feature matching image recognition result.

2. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The car slide rail boundary contour image includes structural boundary lines, contour closed areas, and slide rail connection features. The door area texture distribution image includes texture extension direction, strip arrangement pattern, and surface distribution features. The chain motion trajectory image path includes motion direction lines, trajectory continuity graphics, and chain position sequence. The door deformation part difference image includes boundary offset area, structural deformation position, and time difference layer. The elevator fault feature matching image recognition result includes structural interlacing distribution, abnormal image overlap, and fault feature marking.

3. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The tracking of the movement direction of the chain texture in continuous image frames refers to identifying and matching the temporal changes of the chain surface texture features in continuously captured image frames to verify the movement direction and trajectory of the chain in the time series.

4. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The directional extension method of the door texture refers to analyzing the arrangement trend and directional changes of the texture on the surface of the elevator door in the image, and identifying the dominant extension direction and distribution pattern in space.

5. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Acquire image data frames during elevator operation, perform spatial domain enhancement based on image grayscale distribution characteristics, extract grayscale abnormal regions corresponding to the slide rail connection structure in the image, filter and cluster pixels with grayscale gradient values ​​higher than the preset edge response threshold, and obtain the initial pixel set of the visual region of the slide rail connection structure. S102: Call the initial pixel set of the visual region of the sliding rail connection structure, construct a connection map according to the two-dimensional coordinate relationship of the pixel points, perform morphological gradient processing on the boundary of the connection path, filter the boundary pixel point group with stable gradient direction and amplitude higher than the edge reference value, and obtain the outer edge line segment sequence of the structure region. S103: Call the sequence of line segments on the outer edge of the structure region, determine the connection relationship based on the start and end coordinates of the line segments, filter the combination of line segments that meet the closed loop judgment benchmark value to form a closed region, and generate the boundary contour image content of the car slide rail.

6. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The specific steps of S2 are as follows: S201: Call the content of the car slide rail boundary contour image, filter the set of regions with brightness changes and closed boundaries in the adjacent regions according to the structural boundary position in the image coordinate space, and perform geometric alignment verification on the edge of the region shape and the slide rail contour. After filtering out the regions that do not meet the alignment conditions, obtain the door projection area image fragment. S202: Call the image segment of the projection area of ​​the door body, calculate the concentration of the gradient direction angle of adjacent pixels based on the direction of image gray value change, filter the pixel group with the direction concentration higher than the preset texture direction judgment threshold, and generate a texture direction extension pattern structure. S203: Invoke the texture direction extension pattern structure, and perform directional clustering on the positional relationship in the image space according to the direction vector and distribution density of the texture strips in the guide area to obtain the texture distribution image content of the door area.

7. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The specific steps for S3 are as follows: S301: Call the texture distribution image content of the door area, retrieve the texture area enclosed by the door boundary in the image coordinate space, and filter the sub-regions with texture density higher than the preset chain texture filtering benchmark value based on the texture strip density and direction consistency parameters in the region, and mark the corresponding position in the image to obtain the chain drive area position index set; S302: Call the chain drive region position index set to obtain the image block of the corresponding coordinate region in the continuous image frame, and generate the chain texture direction change sequence according to the angle change trend of the texture direction in the image block and the coordinate displacement of the texture feature point between the continuous frames. S303: Call the chain texture direction change sequence, perform path combination operation based on the continuity of texture position offset vectors between adjacent frames, and perform topological verification on the connection relationship of all path segments. After filtering out non-continuous segments, obtain the chain motion trajectory image path.

8. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The specific steps of S4 are as follows: S401: Call the image path of the chain motion trajectory, extract the image coordinate block within the trajectory coverage area, obtain the set of door boundary pixels in consecutive image frames, perform inter-frame comparison based on the gradient direction of boundary pixels under the same spatial position, and mark the pixel positions where the angle difference is greater than the preset door boundary difference threshold to obtain the boundary structure change mark index set. S402: Call the boundary structure change marker index set, count the time points when the marker index appears in the image frame sequence, extract the temporal information of boundary changes according to the time distribution law, select the image frames whose difference value first exceeds the boundary difference judgment benchmark value as the differential frames, and generate the door boundary difference time node sequence. S403: Call the time node sequence of the door boundary difference, extract the boundary image block of the marked index coverage area in the corresponding image frame, perform pixel alignment and gray level difference operation on the frame images before and after the time node, and construct a visual mapping map based on the gray level difference to obtain the difference image content of the door deformation part.

9. The elevator fault accident identification method based on image recognition according to claim 1, characterized in that, The specific steps of S5 are as follows: S501: Call the difference image content of the deformed part of the door, the image path of the chain movement trajectory and the image content of the boundary contour of the car slide rail, perform pixel alignment operation on the three types of images in a unified image coordinate system, and overlay the image layers according to the layer identifier to obtain the structural information overlay image matrix. S502: Call the structural information to overlay the image matrix, divide the image region into equally spaced grid blocks, extract the texture edge values, direction vectors and number of overlapping markers in each grid block, calculate the proportion of the number of overlapping structures to the total number of boundary pixels, and compare it with the feature overlap index to obtain the image region edge overlap distribution value. S503: Call the edge overlap distribution value of the image region, and according to the distribution difference of density values ​​in the image space, search for regions with density higher than the preset fault identification threshold and mark the coordinates of the center point of the region. Combine the texture pattern of the corresponding region image block to establish a recognition feature set and obtain the elevator fault feature matching image recognition result.

10. An elevator malfunction and accident identification system based on image recognition, characterized in that, The system is used to implement the elevator fault accident identification method based on image recognition as described in any one of claims 1-9, and the system includes: The slide rail boundary extraction module is used to achieve S1: collecting image data during elevator operation, locating the visual region in the image corresponding to the elevator car slide rail connection structure, segmenting the outer edge line segment of the visual region, and outputting the slide rail boundary contour image content; The door texture analysis module is used to implement S2: call the content of the car slide rail boundary contour image, identify the projection area of ​​the elevator door in the image, analyze the directional extension mode of the door texture in the projection area, extract the strip distribution pattern, and output the texture distribution image content of the door area; The chain trajectory tracking module is used to implement S3: call the texture distribution image content of the door area, mark the chain driving area within the door boundary, track the movement direction of the chain texture in continuous image frames, and output the chain motion trajectory image path; The door deformation detection module is used to implement S4: call the chain motion trajectory image path, compare the door boundary within the path coverage area in multiple frames, mark the structural changes at different time points, construct visual comparison graphics, and output the difference image content of the door deformation part; The fault feature recognition module is used to implement S5: call the difference image content of the door body deformation part, the image path of the chain movement trajectory and the image content of the car slide rail boundary contour, compare the edge overlap, and output the elevator fault feature matching image recognition result.

Images