Escalator skirt board and step gap detection method, electronic device and medium

Abstract

The application provides an escalator skirt board and step gap detection method, an electronic device and a medium, and relates to the technical field of visual detection. The method comprises the following steps: acquiring a visual file, wherein the visual file comprises a target image, a first visual image of a previous frame of the target image, and a second visual image of a next frame of the target image; the target image, the first visual image and the second visual image all have visual features for representing an escalator skirt board and step gap; based on a preset algorithm, a definition parameter in the target image is obtained, the definition parameter is used to represent the definition of the target image, and the definition parameter is positively correlated with the definition; and based on the definition parameter, a detection strategy of the gap is determined. The method is suitable for continuous detection of the escalator skirt board and step gap.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual detection, in particular to an escalator skirtboard and step gap detection method, an electronic device and a medium. BACKGROUND

[0002] There is a certain gap between the escalator steps and the skirtboard. If the gap is too large, the passenger's clothes, shoelaces, fingers, etc. are easy to be brought into it during the operation of the escalator, resulting in serious pinching accidents. The TSG T7001-2023 "Elevator Supervision and Periodic Inspection Rules" stipulates that the horizontal gap of any one side should not be greater than 4mm, and the total gap at the symmetrical position of both sides should not be greater than 7mm.

[0003] The traditional manual detection method is to measure the gap between the steps one by one by measuring tools, which has the defects of small measurement space, large detection error, high labor intensity, and low detection efficiency.

[0004] There are also some escalator skirtboard and step gap detection devices in the prior art, but most of them are fixed on the steps for detection. Although the measurement can be completed when the escalator is running, only the gap between a single step and the skirtboard can be measured, and the actual situation of the gap between the escalator skirtboard and the step cannot be reflected completely at one time. SUMMARY

[0005] Therefore, one of the purposes of the embodiments of the present application is to provide an escalator skirtboard and step gap detection method, which can solve the problem that the prior art cannot complete the escalator skirtboard and step gap detection.

[0006] To achieve the above technical purposes, the technical solutions adopted by the present application are as follows:

[0007] In a first aspect, the embodiments of the present application provide an escalator skirtboard and step gap detection method, which comprises:

[0008] Obtaining a visual file, wherein the visual file is obtained based on shooting the gap between the escalator skirtboard and the step, and the visual file includes a target image, a first visual image of a previous frame of the target image, and a second visual image of a next frame of the target image;

[0009] Obtaining a definition parameter in the target image based on a first preset algorithm, wherein the definition parameter is used to represent the definition of the target image, and the definition parameter is positively correlated with the definition;

[0010] Determining the detection strategy of the gap based on the definition parameter, comprising:

[0011] When the definition parameter is greater than a first preset value, the target image is taken as a to-be-detected image; when the definition parameter is less than or equal to the first preset value and greater than a second preset value, the target image is processed based on a preset deblurring strategy to obtain a to-be-detected image; when the definition parameter is less than or equal to the second preset value, the target image, the first visual image and the second visual image are fused based on a preset fusion algorithm to obtain a to-be-detected image.

[0012] Based on a preset detection model, lengths of the first gap and the second gap in the to-be-detected image are obtained, the first gap and the second gap being respectively used to represent the gaps between the two ends of the step and the corresponding apron.

[0013] Further, the preset detection model is a deep learning model, and the deep learning model comprises an input layer, an encoder, a decoder and an output layer.

[0014] The encoder comprises a first branch and a second branch.

[0015] The first branch comprises a convolution layer and an attention channel layer.

[0016] The second branch comprises a down-sampling layer and a full connection layer.

[0017] Based on the preset detection model, the lengths of the gaps in the to-be-detected image are obtained, comprising:

[0018] Based on the to-be-detected image, the input layer is used to normalize pixels of the to-be-detected image to obtain a normalized to-be-detected image, and the normalized to-be-detected image is input into the first branch and the second branch.

[0019] When the first branch receives the normalized to-be-detected image, the convolution layer is used to perform convolution processing on the normalized to-be-detected image to obtain a convolution to-be-detected image.

[0020] The average value of pixels of each channel of the convolution to-be-detected image is calculated, and a weight value is added to the channel based on the average value to obtain a first branch image, wherein when the average value is greater than a pixel preset value, the weight value is located in a first interval, and when the average value is less than or equal to the pixel preset value, the weight value is located in a second interval, and the minimum value of the first interval is greater than the maximum value in the second interval.

[0021] When the second branch receives the normalized to-be-detected image, the normalized to-be-detected image is input into the down-sampling layer, and the down-sampling layer is used to perform down-sampling on the normalized to-be-detected image to obtain a down-sampling to-be-detected image, and the resolution of the down-sampling to-be-detected image is 1 / N of the normalized to-be-detected image, wherein N is a natural number greater than or equal to 2.

[0022] When the full connection layer receives the down-sampled to-be-detected image, the blur angle and the blur length of the to-be-detected image are obtained through the full connection layer;

[0023] After the decoder receives the convolution to-be-detected image, the resolution of the convolution to-be-detected image is increased through upsampling operation, and each pixel point in the upsampling convolution to-be-detected image is corrected based on the blur angle and the blur length, to obtain a corrected to-be-detected image, wherein the resolution of the corrected to-be-detected image is the same as that of the to-be-detected image;

[0024] After the output layer receives the corrected to-be-detected image, the edges of the first gap and the second gap are obtained through the output layer and the weight values of each channel in the corrected to-be-detected image, and the lengths of the first gap and the second gap are obtained based on the edges.

[0025] Further, the to-be-detected image is obtained by fusing the target image, the first visual image and the second visual image based on a preset fusion algorithm, comprising:

[0026] The dense optical flow of the first visual image and the dense optical flow of the second visual image are calculated based on a second preset algorithm, to obtain the first horizontal offset and the first vertical offset of all pixel points in the first visual image, and the second horizontal offset and the second vertical offset of all pixel points in the second visual image, wherein the first horizontal offset, the first vertical offset, the second horizontal offset and the second vertical offset are in the coordinate system of the visual device corresponding to the visual file;

[0027] The median value of all the first horizontal offsets and the median value of all the first vertical offsets are obtained to obtain the first offset of the first visual image relative to the target image, and the median value of all the second horizontal offsets and the median value of all the second vertical offsets are obtained to obtain the second offset of the second visual image relative to the target image;

[0028] Based on the first offset, all pixel points of the first visual image are translated, and based on the second offset, all pixel points of the second visual image are translated, so that the translated first visual image, the target image and the translated second visual image are aligned;

[0029] Each corresponding pixel position in the translated first visual image, the target image and the translated second visual image is traversed, and at each pixel position, the pixel point of the pixel position with the maximum gradient value is selected from the translated first visual image, the target image and the translated second visual image;

[0030] The image to be detected is obtained by fusing the pixels at the positions of all pixels with the largest gradient values.

[0031] Furthermore, the step of processing the target image based on a preset deblurring strategy to obtain the image to be detected includes:

[0032] The target image is input into a CNN backbone network model, and the blur kernel of the target image is obtained through the CNN backbone network model.

[0033] Both the blur kernel and the target image are subjected to Fourier transform to obtain the Fourier transform blur kernel and the Fourier transform target image, respectively.

[0034] Based on the Fourier transform blur kernel and the Fourier transform target image, intermediate variables are obtained according to the Wiener filter;

[0035] The intermediate variable is subjected to inverse Fourier transform to obtain the image to be detected.

[0036] Furthermore, obtaining the sharpness parameters in the target image based on the first preset algorithm includes:

[0037] The target image is processed to obtain a grayscale target image;

[0038] Calculate the gradient value of each pixel in the grayscale target image;

[0039] Calculate the sum of all the gradient values, which is used to characterize the sharpness parameter.

[0040] Further, the step of obtaining the edge of the gap through the output layer and the weight value of each channel in the corrected image after receiving the corrected image at the output layer, and obtaining the length of the gap based on the edge, includes:

[0041] Based on the weight values, a probability map is obtained, which includes the probability that each pixel in the image to be detected is an edge;

[0042] Based on the probability map, the first edge of the first gap and the second edge of the second gap are obtained;

[0043] The first pixel width of the first edge and the second pixel width between the second edges are calculated.

[0044] Based on the first pixel width and the second pixel width, the length of the first gap and the length of the second gap are obtained.

[0045] Furthermore, after determining the detection strategy for the gap based on the sharpness parameter, the method further includes:

[0046] When the length of the first gap is greater than 4 mm, the length of the second gap is greater than 4 mm, or the sum of the length of the first gap and the length of the second gap is greater than 7 mm, a prompt is issued.

[0047] In a second aspect, the embodiments of the present application further provide an electronic device, comprising a processor and a memory coupled to each other, and the memory stores a computer program, when the computer program is executed by the processor, the electronic device executes the method described above.

[0048] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, when the computer program runs on a computer, the computer executes the method described above.

[0049] The application with the technical scheme has the following advantages:

[0050] In the technical scheme provided in the present application, the optimal processing path is selected according to the real-time image quality: the clear image is directly analyzed to ensure efficiency; the single-frame enhancement is performed on the moderately blurred image to balance the effect and computing power; and the time sequence information is fused for the severely blurred image to use multiple frames of data to offset single-frame vibration blur, so that usable input can still be obtained under extreme conditions. The system can cope with the complex working conditions that are inevitable in the operation of the escalator, such as severe vibration, thereby ensuring the continuity and stability of detection. BRIEF DESCRIPTION OF DRAWINGS

[0051] The present application can be further illustrated by the non-limiting embodiments shown in the accompanying drawings. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope, and other related drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 The flowchart provided for the embodiments of the present application.

[0053] Figure 2 The visual device position diagram provided for the embodiments of the present application.

[0054] Figure 3 The sub-flowchart of S120 provided for the embodiments of the present application.

[0055] Figure 4 The first gap and the second gap continuous measurement result diagram provided for the embodiments of the present application. DETAILED DESCRIPTION

[0056] The application will be described in detail below with reference to the drawings and specific embodiments. It should be noted that similar or identical parts are denoted by the same reference numerals in the drawings or description, and the implementation not shown or described in the drawings is in the form known to those skilled in the art. In the description of the application, the terms "first", "second", etc. are only used for differentiation and cannot be understood as indicating or implying relative importance.

[0057] The electronic device provided by the embodiments of the application can include a processing module and a storage module. The storage module stores a computer program, and when the computer program is executed by the processing module, the electronic device can execute the corresponding steps in the escalator skirtboard and step gap detection method.

[0058] Please refer to Figure 1 The application also provides an escalator skirtboard and step gap detection method. The escalator skirtboard and step gap detection method can include the following steps:

[0059] S110, obtaining a visual file, wherein the visual file is obtained based on photographing the escalator skirtboard and step gap, and the visual file includes a target image, a first visual image of a previous frame of the target image, and a second visual image of a next frame of the target image;

[0060] S120, obtaining a definition parameter in the target image based on a first preset algorithm, the definition parameter being used to represent the definition of the target image, and the definition parameter being positively correlated with the definition;

[0061] S130, determining a detection strategy of the gap based on the definition parameter, including:

[0062] When the definition parameter is greater than a first preset value, the target image is taken as a to-be-detected image; when the definition parameter is less than or equal to the first preset value and greater than a second preset value, the target image is processed based on a preset deblurring strategy to obtain the to-be-detected image; and when the definition parameter is less than or equal to the second preset value, the target image, the first visual image and the second visual image are fused based on a preset fusion algorithm to obtain the to-be-detected image;

[0063] Based on a preset detection model, the length of a first gap and the length of a second gap in the to-be-detected image are obtained, and the first gap and the second gap are respectively used to represent the gap between the two ends of the step and the corresponding skirtboard.

[0064] The steps of the escalator skirtboard and step gap detection method will be described in detail as follows:

[0065] In S110, the visual file is obtained based on photographing the escalator skirtboard and step gap as shown in FIG. 1. Figure 2The visual device 200 shown captures images, and the visual files can be video formats such as WMV and ASF. In this embodiment, the visual device 200 is magnetically fixed to the skirt board of the horizontal running section of the escalator, located in front of the intersection line of the anti-pinch device endpoint and the comb teeth and tread surface (the inspection rules require that the anti-pinch device endpoint be located at a position not less than 50mm and not more than 150mm in front of the intersection line of the comb teeth and tread surface). The lens is directly facing the gap between the skirt board and the step. The lens is adjustable to achieve continuous acquisition and gap calculation of the gap image of each step when the escalator is running.

[0066] The visual file includes several frames of images that are consecutive in time. In this embodiment, at a certain moment, one of the images is selected as the target image, the frame before the target image is the first visual image, and the frame after the target image is the second visual image.

[0067] In S120, such as Figure 3 As shown, it includes the following steps:

[0068] S121: Process the target image into grayscale to obtain a grayscale target image;

[0069] S122: Calculate the gradient value of each pixel in the grayscale target image;

[0070] S123: Calculate the sum of all gradient values, which is used to characterize the sharpness parameter.

[0071] In this embodiment, since the target image is a color image, the target image is first processed into grayscale.

[0072] In S122, the Sobel operator is first used to calculate the gradient of the target image in the X and Y directions. For example, a Sobel convolution kernel in the X direction and a Sobel convolution kernel in the Y direction are defined. Then, the grayscale target image is convolved with the Sobel convolution kernels in the X and Y directions respectively to obtain the gradient values ​​of the pixels in the target image. The larger the gradient value, the clearer the pixel. Therefore, in S123, all gradient values ​​are summed. The larger the sum, the clearer the image (it is unlikely that some pixels will be particularly clear while others are particularly blurry). Therefore, the sum represents the sharpness parameter, and the larger the sharpness parameter, the clearer the target image.

[0073] In S130, whether to use the target image for gap detection is determined by the sharpness of the target image. When the sharpness parameter is greater than a first preset value, it means that the target image is relatively clear, and gap detection can be performed based on the target image. When the sharpness parameter is less than or equal to the first preset value and greater than a second preset value, it means that the target image is relatively blurred, and the target image needs to be processed based on a preset deblurring strategy to improve the sharpness of the target image before target detection. When the sharpness parameter is less than the second preset value, it means that the target image is particularly unclear due to vibration, so the target image, the first visual image and the second visual image need to be fused, and the fused image is used as the image for gap detection.

[0074] In this embodiment, the method for obtaining the to-be-detected image by processing the target image through the preset deblurring strategy is as follows:

[0075] The target image is input into the CNN backbone network model, and the blur kernel of the target image is obtained through the CNN backbone network model. The blur kernel is a mathematical model used to describe the image degradation process in digital image processing, which is represented as a square matrix (or operator) for defining the conversion process from high resolution to low resolution. In this step, the CNN backbone network model is a deep learning model used to extract multi-level features from blurred images. The CNN backbone network model not only learns spatial domain features, but also converts feature maps to the frequency domain through fast Fourier transform to extract frequency domain features. The two are fused to allow the network to utilize both texture information in the spatial domain and blur pattern information in the frequency domain. Finally, a global average pooling and a fully connected layer are used to directly regress a fixed-size blur kernel matrix k. The sum of all elements in this matrix is 1 (achieved through Softmax or normalization) to meet the physical definition of the blur kernel.

[0076] The training method of the CNN backbone network model in this embodiment is as follows:

[0077] First, a large number of original images are collected from escalators, covering different light conditions, different degrees of motion blur, vibration, and scenes with or without stains. Label data: corresponding pixel-level label map (ground truth), which is manually annotated to accurately mark the area of "step gap" in the image, usually represented as a binary mask (gap area is 1 and background is 0).

[0078] II. Using end-to-end training method, using labeled data as supervision signal. During training, the original image is input into the network, and the error between the output and the true value labeled image is calculated through the loss function (such as cross-entropy loss, Dice loss). All parameters in the network (including convolutional layer weights and attention layer parameters) are constantly iteratively updated using backpropagation algorithm and gradient descent optimizer, aiming to minimize the loss function, that is, to make the network's prediction results closer and closer to the true value of artificial labeling, so as to teach the network how to ignore the interference of blur and stains and accurately identify the gap features.

[0079] The blurred kernel and the to-be-detected image are subjected to Fourier transform to obtain a Fourier transform blurred kernel and a Fourier transform to-be-detected image. In this step, the blurred kernel and the target image are converted from the spatial domain to the frequency domain through Fourier transform.

[0080] Based on the Fourier transform blurred kernel and the Fourier transform to-be-detected image, an intermediate variable is obtained according to a Wiener filter.

[0081] The Wiener filter in this embodiment is:

[0082] F_L_hat(u,v)=[F_K*(u,v) / (|F_K(u,v)|²+K)]·F_B(u,v)

[0083] (u,v): coordinates (frequency components) in the frequency domain.

[0084] F_L_hat(u,v): estimated complex spectrum value of the clear image at frequency (u,v).

[0085] F_K(u,v): frequency domain representation of the blurred kernel.

[0086] F_K*(u,v): complex conjugate of F_K(u,v).

[0087] |F_K(u,v)|²: power spectrum of the blurred kernel transfer function, equal to F_K*(u,v)·F_K(u,v).

[0088] F_B(u,v): frequency domain representation of the blurred image.

[0089] K: a key parameter, usually a constant, whose value is the inverse of the signal-to-noise ratio (SNR). In the absence of prior knowledge, K is considered as a hyperparameter that needs to be manually adjusted.

[0090] The intermediate variable is subjected to inverse Fourier transform to obtain the to-be-detected image.

[0091] In this embodiment, the manner in which the target image, the first visual image and the second visual image are fused based on the preset fusion algorithm in S130 to obtain the to-be-detected image includes the following steps:

[0092] Based on the second preset algorithm, the dense optical flow of the first visual image and the dense optical flow of the second visual image are calculated.

[0093] According to the dense optical flow of the first visual image and the dense optical flow of the second visual image, the first lateral offset and the first longitudinal offset of all pixel points in the first visual image, and the second lateral offset and the second longitudinal offset of all pixel points in the second visual image are obtained, wherein the first lateral offset, the first longitudinal offset, the second lateral offset and the second longitudinal offset are in the coordinate system of the visual device corresponding to the visual file.

[0094] The median value of all the first lateral offsets and the median value of all the first longitudinal offsets are obtained to obtain the first offset of the first visual image relative to the target image; the median value of all the second lateral offsets and the median value of all the second longitudinal offsets are obtained to obtain the second offset of the second visual image relative to the target image.

[0095] Based on the first offset, all the pixel points of the first visual image are translated, and based on the second offset, all the pixel points of the second visual image are translated, so that the translated first visual image, the target image and the translated second visual image are aligned.

[0096] Each corresponding pixel position in the translated first visual image, the target image and the translated second visual image is traversed, and at each pixel position, the pixel point of the pixel position with the maximum gradient value is selected from the translated first visual image, the target image and the translated second visual image.

[0097] All the pixel points of the pixel positions with the maximum gradient value are fused to obtain the to-be-detected image.

[0098] For each pixel coordinate (i, j) in the image: compare the gradient values of the three aligned images (the translated first visual image, the target image and the translated second visual image) at the same coordinate position (i.e. each corresponding pixel point) (i, j). Then, the color value (or gray value) of the pixel point with the maximum gradient value is selected as the pixel value of the output image at the position (i, j).

[0099] By traversing each pixel position in the image and repeating the above selection operation, a completely new fused image with the same resolution as the original image can be finally generated. This process ensures that the output image has the same number of pixels as the original image, and at the same time, the image quality is improved by selecting the clearest pixel information. Figure 1

[0100] ​In this embodiment, the dense optical flow of the first visual image is calculated as the motion vector of each pixel between the target image and the first visual image. The dense optical flow of the second visual image is calculated as the motion vector of each pixel between the target image and the second visual image.

[0101] The second preset algorithm of this embodiment refers to the calculation method of calculating the dense optical flow of the first visual image and the dense optical flow of the second visual image. The second preset algorithm is based on the assumptions of brightness constancy, time continuity or spatial consistency, etc. to establish the corresponding relationship between the pixels of the continuous frames, so as to solve the motion vector of each pixel.

[0102] When the second preset algorithm calculates the dense optical flow, the target image is taken as the reference to process the first visual image and the second visual image respectively. The core is to establish a mathematical model of pixel motion based on the brightness constancy assumption (the brightness of the same pixel does not change in continuous frames), the time continuity assumption (the motion amplitude of adjacent pixels is small) and the spatial consistency assumption (the motion trend of adjacent pixels is similar). For the first visual image, the algorithm analyzes the brightness change of the first visual image and the target image pixel by pixel, constructs the optical flow equation by calculating the brightness gradient (including horizontal and vertical gradient) of the local area and combining the brightness change rate in the time dimension. Since the optical flow equation of a single pixel has multiple solutions, the algorithm uses the spatial consistency assumption to impose a smoothing constraint on the motion vector of adjacent pixels to form an overdetermined equation set, and then solves it by using the least square method and other optimization methods to obtain the first horizontal displacement and the first vertical displacement of each pixel, i.e. the dense optical flow of the first visual image. The processing logic of the second visual image is the same as that of the first visual image. By analyzing the local brightness relationship between the second visual image and the target image, combining the spatial smoothing constraint to solve the equation set, the second horizontal displacement and the second vertical displacement of each pixel are obtained, and the dense optical flow of the second visual image is formed. The whole process continuously adjusts the motion vector through iterative optimization until the preset error threshold is met, and finally outputs the motion vector field of all pixels in the two images relative to the target image.

[0103] The calculation of the first lateral offset, the first longitudinal offset, the second lateral offset and the second longitudinal offset is based on the decomposition of the pixel motion vector in dense optical flow analysis. With the target image as a reference, for the first visual image, the preset algorithm first calculates the brightness gradient of the local area (the lateral gradient represents the brightness change rate of the pixel in the horizontal direction, and the longitudinal gradient represents the vertical change rate), combines the time brightness change between the two frames, and constructs the basic equation of the optical flow. Then, by using the spatial consistency constraint (the motion trend of adjacent pixels is similar), the over-determined equation set is solved by an optimization algorithm to obtain the motion vector of each pixel relative to the target image. The projection component of the vector on the horizontal axis (lateral) of the visual device coordinate system is the first lateral offset, and the projection component on the vertical axis (longitudinal) is the first longitudinal offset. For the second visual image, the calculation logic is consistent with that of the first visual image: after analyzing the local brightness relationship between the target image, the motion vector of each pixel is solved, and the vector is decomposed into the lateral and longitudinal axes of the visual device coordinate system to obtain the second lateral offset (horizontal displacement) and the second longitudinal offset (vertical displacement), respectively. These offsets quantify the position changes of each pixel in the two visual images relative to the target image in the lateral and longitudinal directions, and are based on the visual device coordinate system, ensuring that the displacement data is comparable and subsequent processing is in the same spatial scale.

[0104] In S130 of the embodiment, the preset detection model is a deep learning model, and the deep learning model includes an input layer, an encoder, a decoder, and an output layer; the encoder includes a first branch and a second branch; the first branch includes a convolution layer and an attention channel layer; and the second branch includes a down-sampling layer and a full connection layer.

[0105] In the embodiment, S150 can specifically include the following steps:

[0106] Based on the to-be-detected image, the to-be-detected image pixels are normalized through the input layer to obtain a normalized to-be-detected image, and the normalized to-be-detected image is input into the first branch and the second branch;

[0107] When the first branch receives the normalized to-be-detected image, the normalized to-be-detected image is subjected to convolution processing based on the convolution layer to obtain a convolution to-be-detected image;

[0108] The average value of the pixels of each channel of the convolution to-be-detected image is calculated, and a weight value is added to the channel based on the average value to obtain a first branch image, wherein when the average value is greater than a pixel preset value, the weight value is in a first interval, and when the average value is less than or equal to the pixel preset value, the weight value is in a second interval, and the minimum value of the first interval is greater than the maximum value in the second interval;

[0109] When the second branch receives the normalized to-be-detected image, the normalized to-be-detected image is input to a down-sampling layer, the normalized to-be-detected image is down-sampled by the down-sampling layer, and a down-sampled to-be-detected image is obtained, the resolution of the down-sampled to-be-detected image is 1 / N of the normalized to-be-detected image, wherein N is a natural number greater than or equal to 2;

[0110] When the full connection layer receives the down-sampled to-be-detected image, the blur angle and the blur length of the to-be-detected image are obtained by the full connection layer. The blur angle represents the direction or angle of the motion blur. When the to-be-detected image is blurred due to elevator vibration or the like, the blur is not chaotic, but extends in a dominant direction. The blur angle represents this direction. In this embodiment, the horizontal right direction is taken as the blur angle of 0°, and the counterclockwise rotation is taken as the positive angle. The unit of the blur length is pixel, which defines a normal pixel. Due to the blur caused by vibration or the like, the distance of the elongated pixel is defined. For example, the blur length is equal to 5, which means that the information of each pixel is evenly distributed on a line segment with a length of 5 pixels during the blur process.

[0111] When the decoder receives the convolution to-be-detected image, the convolution to-be-detected image is up-sampled to increase the resolution of the convolution to-be-detected image, and each pixel in the up-sampled convolution to-be-detected image is corrected based on the blur angle and the blur length, to obtain a corrected to-be-detected image. The resolution of the corrected to-be-detected image is the same as that of the to-be-detected image. For this step, the way to correct each pixel in the up-sampled convolution to-be-detected image according to the blur angle and the blur length can be as follows: first, for each pixel in the up-sampled image, taking the pixel as the core, the range of pixel information propagation (i.e., the pixel area affected by the blur) is determined according to the known blur length, and then the propagation direction of the pixel information in the range is determined according to the blur angle (for example, the information propagates along the horizontal direction when the blur angle is 0°, and propagates along the diagonal direction when the blur angle is 45°); then, the original information source of the pixel during the blur process is traced back, that is, the original pixel position corresponding to the “blur length” from the current pixel in the blur direction is extracted, and the effective feature information of the original position in the convolution to-be-detected image is extracted; finally, the original feature information extracted is used to correct the numerical value of the current up-sampled pixel, to make up for the information loss or blur residue that may be caused by the up-sampling, so that each corrected pixel can match the real information distribution under the original resolution of the to-be-detected image, and finally the corrected to-be-detected image is consistent with the initial to-be-detected image in resolution, and the targeted repair of the blur distortion is realized.

[0112] When the decoder receives the convolution to-be-detected image, the convolution to-be-detected image is up-sampled to increase the resolution of the convolution to-be-detected image, and each pixel in the up-sampled convolution to-be-detected image is corrected based on the blur angle and the blur length, to obtain a corrected to-be-detected image. The resolution of the corrected to-be-detected image is the same as that of the to-be-detected image. For this step, the way to correct each pixel in the up-sampled convolution to-be-detected image according to the blur angle and the blur length can be as follows: first, for each pixel in the up-sampled image, taking the pixel as the core, the range of pixel information propagation (i.e., the pixel area affected by the blur) is determined according to the known blur length, and then the propagation direction of the pixel information in the range is determined according to the blur angle (for example, the information propagates along the horizontal direction when the blur angle is 0°, and propagates along the diagonal direction when the blur angle is 45°); then, the original information source of the pixel during the blur process is traced back, that is, the original pixel position corresponding to the “blur length” from the current pixel in the blur direction is extracted, and the effective feature information of the original position in the convolution to-be-detected image is extracted; finally, the original feature information extracted is used to correct the numerical value of the current up-sampled pixel, to make up for the information loss or blur residue that may be caused by the up-sampling, so that each corrected pixel can match the real information distribution under the original resolution of the to-be-detected image, and finally the corrected to-be-detected image is consistent with the initial to-be-detected image in resolution, and the targeted repair of the blur distortion is realized.

[0113] In this embodiment, the asymmetric dual-branch encoder is used to process the static texture details and dynamic motion blur in the image respectively and cooperatively, and finally a precise decoder is used to realize the pixel-level precise positioning. The pixel preset value of this embodiment is set based on experience by those skilled in the art based on the above method.

[0114] In this embodiment, after receiving the corrected to-be-detected image, the output layer and the weight value of each channel in the corrected to-be-detected image are used to obtain the edge of the gap, and based on the edge, the length of the gap is obtained, including:

[0115] Based on the weight value, a probability map is obtained, and the probability map includes the probability that each pixel point in the to-be-detected image is an edge;

[0116] Based on the probability map, a first edge of a first gap and a second edge of a second gap are obtained;

[0117] The first pixel width of the first edge and the second pixel width between the second edges are calculated;

[0118] Based on the first pixel width and the second pixel width, the length of the first gap and the length of the second gap are obtained.

[0119] In this embodiment, the probability map is a feature map generated in combination with the weight values of each channel of the corrected to-be-detected image, and its core function is to quantify the possibility of each pixel point in the image belonging to the "edge". The size of the probability map is consistent with the corrected to-be-detected image, and the numerical value (usually between 0 and 1) of each pixel point represents the probability that the position is a gap edge - the closer the value is to 1, the higher the possibility that the pixel belongs to the edge; the closer the value is to 0, the higher the possibility that the pixel belongs to the background or non-edge region. Through the probability map, potential edge pixels can be distinguished from continuous probability distribution, providing a quantitative basis for subsequent extraction of gap edges. The pixel width is an index used to describe the spatial size of the edge or gap in the image, and is measured in pixels.

[0120] The first gap and the second gap represent the gaps between the two sides of the step and the apron plate, that is, the first gap corresponds to the gap between one side of the step and the apron plate, and the second gap corresponds to the gap between the other side of the step and the apron plate. The first pixel width refers to the vertical distance in the image corresponding to the number of pixels of the two edge lines of the first gap, that is, the edge of the step and the edge of the apron plate. The second pixel width refers to the vertical distance in the image corresponding to the number of pixels of the two edge lines of the second gap, that is, the other side edge of the step and the edge of the apron plate. The pixel width of the gap reflects the size scale of the gap in the image. For example, if the two edge lines of the first gap are separated by 4 pixels, the first pixel width is 4. Subsequently, the pixel width can be converted into the real physical length of the gap by combining the physical scale of the image obtained through camera calibration (such as how many millimeters each pixel corresponds to).

[0121] The method forms a complete pipeline from input images to generate probability maps, and then extract edges and calculate lengths. The attention mechanism (weight value) in the network directly optimizes the quality of the probability map by suppressing irrelevant features (such as stains) and enhancing key features (such as edges), thereby indirectly but greatly improving the accuracy of the final length measurement.

[0122] In the embodiment, after determining the detection strategy of the gap based on the definition parameter, the method further includes S140, S140 includes:

[0123] When the length of the first gap is greater than 4mm, the length of the second gap is greater than 4mm, or the sum of the length of the first gap and the length of the second gap is greater than 7mm, a prompt is issued.

[0124] When the length of the first gap is greater than 4mm, the length of the second gap is greater than 4mm, or the sum of the length of the first gap and the length of the second gap is greater than 7mm, a prompt is issued.

[0125] The prompt in the embodiment can be an alarm sound or other prompt means.

[0126] Based on the method of the embodiment, the first gap (left) and the second gap (right) are continuously detected at the same time, and the measurement results are as shown in Figure 4 .

[0127] In the embodiment, the processing module can be an integrated circuit chip having a signal processing capability. The processing module can be a general processor. For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0128] The storage module can be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, and the like.

[0129] It should be noted that, for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding process of each step in the foregoing method, and will not be described in detail here.

[0130] The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program. When the computer program runs on a computer, the computer executes the escalator apron and step gap detection method in the above embodiment.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by means of software and necessary general hardware platform. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0132] In the embodiments of the present disclosure, it should be understood that the disclosed method can also be implemented in other ways. The above described method embodiments are only schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the method and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that each block in the flowcharts and / or block diagrams, and the combination of blocks in the flowcharts and / or block diagrams, can be implemented by a dedicated hardware-based system, or can be implemented by a combination of special-purpose hardware and computer instructions. In addition, the functional modules in the various embodiments of the present disclosure can be integrated together to form a separate part, or can exist independently, or two or more modules can be integrated to form a separate part.

[0133] The above only describes the embodiments of the present disclosure and is not used to limit the protection scope of the present disclosure. For those skilled in the art, the present disclosure can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for detecting the gap between the skirt panel and the steps of an escalator, characterized in that, The method includes: A visual file is obtained, wherein the visual file is based on a photograph of the gap between the skirt board and the steps of an escalator, and the visual file includes a target image, a first visual image of the frame preceding the target image, and a second visual image of the frame following the target image; Based on a first preset algorithm, a sharpness parameter in the target image is obtained. The sharpness parameter is used to characterize the sharpness of the target image, and the sharpness parameter is positively correlated with the sharpness. Based on the resolution parameters, a detection strategy for the gap is determined, including: When the sharpness parameter is greater than a first preset value, the target image is used as the image to be detected; when the sharpness parameter is less than or equal to the first preset value but greater than a second preset value, the target image is processed based on a preset deblurring strategy to obtain the image to be detected; when the sharpness parameter is less than or equal to the second preset value, the target image, the first visual image, and the second visual image are fused based on a preset fusion algorithm to obtain the image to be detected. Based on a preset detection model, the lengths of the first gap and the second gap in the image to be detected are obtained. The first gap and the second gap are used to characterize the gaps between the two ends of the ladder and the corresponding apron board.

2. The method according to claim 1, characterized in that, The preset detection model is a deep learning model, which includes an input layer, an encoder, a decoder, and an output layer. The encoder includes a first branch and a second branch; The first branch includes convolutional layers and attention channel layers; The second branch includes a downsampling layer and a fully connected layer; The step of obtaining the gap length in the image to be detected based on a preset detection model includes: Based on the image to be detected, the pixels of the image to be detected are normalized through the input layer to obtain a normalized image to be detected, and the normalized image to be detected is input into the first branch and the second branch. After the first branch receives the normalized image to be detected, it performs convolution processing on the normalized image to be detected based on the convolutional layer to obtain the convolutional image to be detected. Calculate the average pixel value of each channel of the convolutional image to be detected, and add weight values ​​to the channels based on the average value to obtain a first branch image. When the average value is greater than a preset pixel value, the weight value is located in a first interval. When the average value is less than or equal to the preset pixel value, the weight value is located in a second interval. The minimum value of the first interval is greater than the maximum value of the second interval. When the second branch receives the normalized image to be detected, it inputs it to the downsampling layer. The downsampling layer downsamples the normalized image to be detected to obtain the downsampled image to be detected. The resolution of the downsampled image to be detected is 1 / N of the normalized image to be detected, where N is a natural number greater than or equal to 2. After the fully connected layer receives the downsampled image to be detected, the blur angle and blur length of the image to be detected are obtained through the fully connected layer. After receiving the convolutional image to be detected, the decoder performs an upsampling operation on the convolutional image to increase its resolution, and corrects each pixel in the upsampled convolutional image to be detected based on the blur angle and blur length to obtain a corrected image to be detected, wherein the corrected image to be detected has the same resolution as the original image to be detected. After receiving the corrected image to be detected in the output layer, the edges of the first gap and the second gap are obtained through the output layer and the weight value of each channel in the corrected image to be detected, and the lengths of the first gap and the second gap are obtained based on the edges.

3. The method according to claim 1, characterized in that, The process of fusing the target image, the first visual image, and the second visual image based on a preset fusion algorithm to obtain the image to be detected includes: Based on the second preset algorithm, the dense optical flow of the first visual image and the dense optical flow of the second visual image are calculated. Based on the dense optical flow of the first visual image and the dense optical flow of the second visual image, the first horizontal offset and the first vertical offset of all pixels in the first visual image, and the second horizontal offset and the second vertical offset of all pixels in the second visual image are obtained, wherein the first horizontal offset, the first vertical offset, the second horizontal offset and the second vertical offset are all in the coordinate system of the visual device corresponding to the visual file. The median value of all first horizontal offsets and the median value of all first vertical offsets are obtained to obtain the first offset of the first visual image relative to the target image; the median value of all second horizontal offsets and the median value of all second vertical offsets are obtained to obtain the second offset of the second visual image relative to the target image. Based on the first offset, all pixels of the first visual image are translated, and based on the second offset, all pixels of the second visual image are translated, so that the translated first visual image, the target image, and the translated second visual image are aligned. Traverse each corresponding pixel position in the translated first visual image, the target image, and the translated second visual image, and at each pixel position, select the pixel with the largest gradient value from the translated first visual image, the target image, and the translated second visual image; The image to be detected is obtained by fusing the pixels at the positions of all pixels with the largest gradient values.

4. The method according to claim 1, characterized in that, The process of processing the target image based on a preset deblurring strategy to obtain the image to be detected includes: The target image is input into a CNN backbone network model, and the blur kernel of the target image is obtained through the CNN backbone network model. Both the blur kernel and the target image are subjected to Fourier transform to obtain the Fourier transform blur kernel and the Fourier transform target image, respectively. Based on the Fourier transform blur kernel and the Fourier transform target image, intermediate variables are obtained according to the Wiener filter; The intermediate variable is subjected to inverse Fourier transform to obtain the image to be detected.

5. The method according to claim 1, characterized in that, The step of obtaining the sharpness parameters in the target image based on the first preset algorithm includes: The target image is processed to obtain a grayscale target image; Calculate the gradient value of each pixel in the grayscale target image; Calculate the sum of all the gradient values, which is used to characterize the sharpness parameter.

6. The method according to claim 2, characterized in that, After receiving the corrected image to be detected at the output layer, the edge of the gap is obtained through the output layer and the weight value of each channel in the corrected image to be detected, and the length of the gap is obtained based on the edge, including: Based on the weight values, a probability map is obtained, which includes the probability that each pixel in the image to be detected is an edge; Based on the probability map, the first edge of the first gap and the second edge of the second gap are obtained; The first pixel width of the first edge and the second pixel width between the second edges are calculated. Based on the first pixel width and the second pixel width, the length of the first gap and the length of the second gap are obtained.

7. The method according to claim 6, characterized in that, After determining the detection strategy for the gap based on the sharpness parameter, the method further includes: A prompt will be issued when the length of the first gap is greater than 4mm, the length of the second gap is greater than 4mm, or the sum of the lengths of the first gap and the second gap is greater than 7mm.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory coupled together, the memory storing a computer program that, when executed by the processor, causes the electronic device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-7.

Images