Landslide detection method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a landslide detection method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: identifying a plurality of signboard images in images collected for a slope area; the signboard image represents a signboard arranged in the slope area, and bright color blocks and dark color blocks in the signboard are alternately arranged; determining a target reference position for a signboard image in the plurality of signboard images based on the position of a boundary feature pixel at a bright and dark boundary in the signboard image; when an offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, matching an edge image corresponding to the signboard image with an edge template image pre-configured for the signboard representing the signboard image; and when the obtained matching score shows that the signboard represented by the signboard image deviates, determining a landslide detection result for the slope area based on the deviation starting time when the signboard represented by the signboard image begins to deviate. By adopting the method, hardware resource consumption can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of landslide detection, and in particular to a landslide detection method, device, computer equipment, storage medium and program product. Background Art

[0002] A slope is a geological body with a certain slope on one side, which can be composed of natural rock and soil or artificial fill. The stability of the slope is affected by many factors, such as rock and soil properties, groundwater, rainfall, earthquakes, etc. When the slope is unstable, it may cause landslide disasters, which will lead to road interruption, traffic congestion, and even cause property losses and casualties. Therefore, it is of great significance to detect landslides on the slope. In traditional technology, sensors are usually installed on the slope to detect whether a landslide has occurred on the slope.

[0003] However, the method of installing sensors on the slope to detect whether a landslide has occurred requires laying more sensors on the slope and adding supporting equipment for each sensor, such as a signal transmission module, a power supply module, etc., which consumes a lot of hardware resources. Summary of the invention

[0004] Based on this, it is necessary to provide a landslide detection method, device, computer equipment, storage medium and program product that can reduce hardware resource consumption in response to the above technical problems.

[0005] In a first aspect, the present application provides a landslide detection method, comprising:

[0006] Identify multiple signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged;

[0007] For a signboard image among the plurality of signboard images, determining a target reference position representing a position of the signboard image in the image based on a position of a boundary feature pixel at a light-dark boundary in the signboard image;

[0008] Acquire a historical reference position of the sign represented by the sign image in the historical reference image;

[0009] In the case where the offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, an edge map obtained by edge detection based on the sign image is matched with an edge template map pre-configured for the sign represented by the sign image to obtain a matching score;

[0010] When the matching score indicates that the sign represented by the sign image has shifted, a landslide detection result for the slope area is determined based on a shift start time when the sign represented by the sign image starts to shift.

[0011] In a second aspect, the present application also provides a landslide detection device, comprising:

[0012] A recognition module, used to recognize a plurality of signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged;

[0013] A detection module is used to determine, for a signboard image among the multiple signboard images, a target reference position representing the position of the signboard image in the image based on the position of a boundary feature pixel at a light-dark boundary in the signboard image; obtain a historical reference position of the signboard represented by the signboard image in a historical reference image; and, when an offset index value determined based on the target reference position and the historical reference position satisfies a preset offset condition, match an edge map obtained by edge detection based on the signboard image with an edge template map preconfigured for the signboard represented by the signboard image to obtain a matching score; when the matching score indicates that the signboard represented by the signboard image has shifted, determine a landslide detection result for the slope area based on a shift start time when the signboard represented by the signboard image begins to shift.

[0014] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0015] Identify multiple signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged;

[0016] For a signboard image among the plurality of signboard images, determining a target reference position representing a position of the signboard image in the image based on a position of a boundary feature pixel at a light-dark boundary in the signboard image;

[0017] Acquire a historical reference position of the sign represented by the sign image in the historical reference image;

[0018] In the case where the offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, an edge map obtained by edge detection based on the sign image is matched with an edge template map pre-configured for the sign represented by the sign image to obtain a matching score;

[0019] When the matching score indicates that the sign represented by the sign image has shifted, a landslide detection result for the slope area is determined based on a shift start time when the sign represented by the sign image starts to shift.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0021] Identify multiple signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged;

[0022] For a signboard image among the plurality of signboard images, determining a target reference position representing a position of the signboard image in the image based on a position of a boundary feature pixel at a light-dark boundary in the signboard image;

[0023] Acquire a historical reference position of the sign represented by the sign image in the historical reference image;

[0024] In the case where the offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, an edge map obtained by edge detection based on the sign image is matched with an edge template map pre-configured for the sign represented by the sign image to obtain a matching score;

[0025] When the matching score indicates that the sign represented by the sign image has shifted, a landslide detection result for the slope area is determined based on a shift start time when the sign represented by the sign image starts to shift.

[0026] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0027] Identify multiple signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged;

[0028] For a signboard image among the plurality of signboard images, determining a target reference position representing a position of the signboard image in the image based on a position of a boundary feature pixel at a light-dark boundary in the signboard image;

[0029] Acquire a historical reference position of the sign represented by the sign image in the historical reference image;

[0030] In the case where the offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, an edge map obtained by edge detection based on the sign image is matched with an edge template map pre-configured for the sign represented by the sign image to obtain a matching score;

[0031] When the matching score indicates that the sign represented by the sign image has shifted, a landslide detection result for the slope area is determined based on a shift start time when the sign represented by the sign image starts to shift.

[0032] The above-mentioned landslide detection method, device, computer equipment, storage medium and program product, in which a signboard is set in the slope area, can identify multiple signboard images from the image collected for the slope area. Since the light blocks and dark blocks in the signboard are arranged alternately, there are boundary feature pixels at the light and dark boundaries in the signboard image. Based on the position of the boundary feature pixels, the target reference position representing the position of the signboard image in the image can be determined. When the offset index value determined based on the target reference position and the historical reference position meets the preset offset condition, since the historical reference position is the signboard represented by the signboard image in the historical reference image, it can be preliminarily determined that the signboard may have shifted. Furthermore, the edge map obtained by edge detection based on the signboard image is compared with the edge map for the signboard image. The image is matched with a pre-configured edge template map of a signboard represented by the image to obtain a matching score. The matching score can be used to accurately determine whether the signboard is offset. When the matching score determines that the signboard is offset, the landslide detection result for the slope area is further determined based on the offset start time when the signboard begins to offset. This can exclude the situation where the signboard is accidentally offset. In this way, since multiple levels of judgment are performed on the signboard, the landslide detection result obtained is more accurate. Moreover, by setting a signboard with light blocks and dark blocks arranged alternately in the slope area, collecting images of the slope area, and then performing a series of data processing to perform landslide detection, there is no need to lay multiple sensors and sensor support equipment in the slope area, which can reduce hardware resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A schematic diagram of a flow chart of a landslide detection method in one embodiment;

[0035] Figure 2 A schematic diagram of a sample of a close-up sign in an embodiment;

[0036] Figure 3 A schematic diagram of a prospect signboard sample in an embodiment;

[0037] Figure 4 A schematic diagram of a landslide detection scenario in an embodiment;

[0038] Figure 5 A schematic diagram of a landslide detection process in an embodiment;

[0039] Figure 6 is a structural block diagram of a landslide detection device in one embodiment;

[0040] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In an exemplary embodiment, Figure 1 As shown, a landslide detection method is provided. This embodiment uses the method applied to a camera as an example. It can be understood that the method can also be applied to a system including a camera and a server, and the camera collects images of the slope area, and the camera transmits the collected images to the server, and the server executes the method, wherein the server can also be replaced by a terminal. In this embodiment, the method includes the following steps 102 to 110. Among them:

[0043] Step 102, identifying a plurality of signboard images in the image collected for the slope area; the signboard image represents the signboard set in the slope area, and the light blocks and the dark blocks in the signboard are arranged alternately.

[0044] Among them, the slope area is an area in the slope where there is a risk of landslide. The slope can be a slope on one side of a road or a slope near a residence. The slope area can be a part of the slope or the entire area of ​​the slope. The image collected for the slope area can be collected by a camera. The camera can be a high-definition camera. The camera can be set in an area outside the slope area without the risk of collapse. The area covered by the camera's field of view can include a position in the slope whose distance to the camera is within a preset distance range, and the area covered by the camera's field of view can include the slope area, and the preset distance range can be, for example, 10 meters to 300 meters.

[0045] The camera can record a video of the slope area in real time, and the image collected for the slope area can be a video frame in the video recorded by the camera, and specifically can be a video frame obtained at a preset time interval. The preset time interval can be 1 second, 1 minute, 10 minutes, 1 hour or other, which can be set as required. The camera can also shoot an image of the slope area to obtain an image collected for the slope area.

[0046] The signboard image is an image block in the image collected for the slope area. The signboard image may include the actual image area occupied by the signboard when it is presented in the image. It can be understood that one signboard image represents one signboard. Multiple signboards may be set in the slope area, and multiple signboards may be set at intervals in the slope area. When setting a camera and multiple signboards, the front faces of the multiple signboards may be adjusted to face the camera, and the images presented by the multiple signboards in the camera's picture do not block each other, and there are no objects blocking the line of sight when the camera shoots each signboard.

[0047] The signboard may be supported by a support rod, and the support rod may be embedded in the slope area so that the signboard deflects with the deflection of the slope area. The signboard may be formed by covering the light color block and the dark color block on the metal panel. For example, the light color block and the dark color block may be pasted to the metal panel separately, a pattern of alternating light color blocks and dark color blocks may be pasted to the metal panel, and the light color block and the dark color block may be sprayed on the metal panel.

[0048] The light block is an area with bright colors, and the dark block is an area with dim colors. The brightness of the color of the light block is higher than the brightness of the color of the dark block. There can be many combinations of the colors of the light block and the dark block. For example, the color of the light block can be white, and the color of the dark block can be black; for another example, the color of the light block can be white, and the color of the dark block can be green; for another example, the color of the light block can be yellow, and the color of the dark block can be black. The shape of the light block and the dark block can be a rectangle, and the side length of the light block and the dark block can be within a preset length range, such as 40 cm to 120 cm. The number of light blocks and dark blocks in the signboard can be the same.

[0049] Light blocks and dark blocks are arranged alternately, which means that light blocks and dark blocks are arranged interspersed with each other. In the signboard, light blocks and dark blocks can be arranged regularly, for example, they can be arranged in a way that one of every two adjacent blocks is a light block and the other is a dark block, or four blocks can be arranged in a group in the order of light block, dark block, dark block, and light block.

[0050] In the signboard, the light blocks and the dark blocks may also be arranged irregularly. For example, the light blocks and the dark blocks may be arranged randomly when at least a preset number of boundary feature pixels exist in the signboard image corresponding to the signboard. The preset number may be set according to the total number of light blocks and dark blocks expected to be set in the signboard. For example, the preset number may be determined based on a first preset ratio of the total number of light blocks and dark blocks. For example, the total number of light blocks and dark blocks may be 16, the first preset ratio may be 50%, and the preset number may be 8.

[0051] For example, the camera may acquire images of the slope area at preset time intervals, and identify a plurality of signboard images from the acquired images.

[0052] In one embodiment, the camera may input an image captured for the slope area into a trained sign detection model, obtain multiple target frames detected in the image output by the sign detection model, and respectively determine the image blocks identified by each of the multiple target frames in the image as sign images, so as to identify the multiple sign images in the image captured for the slope area.

[0053] Among them, the sign detection model can be obtained by training the target detection model to be trained, such as ‌YOLOv5s (a version of the YOLO (You Only Look Once) series of target detection models), FasterR-CNN (Region-based Convolutional Neural Network), or others. The target box can surround the actual image area occupied by the sign when it is presented in the image. The sign detection model can output the rectangular coordinate information of the target box, and the image block identified by the target box can be determined through the rectangular coordinate information. The rectangular coordinate information may include the upper left corner coordinates and the lower right corner coordinates of the target box, or include the center point coordinates of the target box, the width of the target box, and the height of the target box.

[0054] Step 104 , for a sign image among the plurality of sign images, based on the position of a boundary feature pixel at a light-dark boundary in the sign image, determine a target reference position representing the position of the sign image in the image.

[0055] Among them, for the signboard image in the multiple signboard images, it can be for each signboard image in the multiple signboard images, or it can be for the target signboard image in the multiple signboard images. The target signboard image can be the signboard image selected in response to the signboard selection operation, and can be the signboard image whose clarity meets the clarity condition among the multiple signboard images. The signboard selection operation can be an operation of selecting from the acquired signboard images. The clarity condition is a condition for determining that the clarity of the signboard image is high. The clarity can be measured by the average gradient amplitude of each pixel in the signboard image. The larger the average gradient amplitude, the higher the clarity. The clarity can also be measured by the grayscale variance of the signboard image. The larger the grayscale variance, the higher the clarity. For example, the clarity condition can be that the average gradient amplitude of each pixel in the signboard image is greater than the preconfigured gradient amplitude, or that the grayscale variance of the signboard image is greater than the preconfigured grayscale variance. The preconfigured gradient amplitude is, for example, 50. The preconfigured grayscale variance is, for example, 100.

[0056] The light-dark boundary represents the boundary between adjacent light blocks and dark blocks in the sign represented by the sign image. Each sign image may include one or more light-dark boundaries. The shape of the light blocks and the dark blocks may be a rectangle, and the light-dark boundary may be the pixel edge of the boundary edge between the adjacent light blocks and the dark blocks in the sign presented in the image. The boundary edge in the sign is the common edge between the adjacent light blocks and the dark blocks.

[0057] The boundary feature pixel is a pixel with light-dark boundary characteristics. The boundary feature pixel can be a random pixel in the light-dark boundary, a pixel in the middle of the light-dark boundary, or a pixel determined by different light-dark boundaries. Specifically, it can be a common pixel of a target light-dark boundary pair in different light-dark boundaries. The target light-dark boundary pair includes two target light-dark boundaries, each target light-dark boundary can represent a target boundary edge in the signboard, and the two target boundary edges represented by the target light-dark boundary pair intersect in the signboard, and the angle between the two target boundary edges meets the preset angle condition, then the common pixel of the target boundary pair can represent the intersection of the two target boundary edges represented by the target light-dark boundary pair, that is, the boundary feature pixel can represent the intersection of the two target boundary edges, and the boundary feature pixel can specifically be a pixel presented in the image at the intersection of the two target boundary edges. In the case where the shape of the light block and the dark block is a rectangle, the preset angle condition is that the angle between the two target boundary edges is a right angle.

[0058] Exemplarily, the camera may detect the position of a boundary feature pixel at the boundary between light and dark in a signboard image among multiple signboard images. When the number of boundary feature pixels is one, the position of the boundary feature pixel may be determined as a target reference position representing the position of the signboard image in the image. When the number of boundary feature pixels is multiple, the position of one boundary feature pixel may be selected from the positions of multiple boundary feature pixels, or a position calculated based on the positions of multiple boundary feature pixels may be used as a target reference position representing the position of the signboard image in the image.

[0059] Step 106: Obtain the historical reference position of the sign represented by the sign image in the historical reference image.

[0060] Among them, the historical reference image is a historical image for reference collected before the target acquisition time. The target acquisition time is the acquisition time of the image collected for the slope area in step 102. The historical reference image can specifically be any historical image collected before the target acquisition time, can be a historical image collected at a time point before the target acquisition time and at a preset interval to the target acquisition time, or can be a preset calibration image. The preset interval is such as 5 minutes, 10 minutes, 1 hour or others. The preset calibration image is used to represent the initial state of the slope area. The preset calibration image can be obtained based on the initial image collected for the slope area after the construction placement is completed and before it is put into use. The construction placement may include the placement of signboards and cameras. Putting into use means starting to detect whether a landslide occurs in the slope area. The calibration image can be obtained after image preprocessing of the initial image. Image preprocessing includes image denoising, clarity improvement, brightness balancing and other processing.

[0061] The historical reference position represents the position of the historical image of the sign in the historical reference image. The historical image of the sign is the image of the sign represented by the sign in the historical reference image. It can be understood that the sign image here refers to the sign image targeted by step 104. The method for determining the historical reference position can be the same as the method for determining the target reference position. When the historical reference image is a preset calibration image, the historical reference position can be called the initial reference position, the historical image of the sign can be called the sign reference image, and the initial reference position can represent the position of the sign reference image in the calibration image.

[0062] Exemplarily, the camera may obtain a historical reference position of a sign represented by a pre-stored image of the sign in a historical reference image.

[0063] In one embodiment, the camera may obtain a historical image of the sign in a pre-stored historical reference image, and determine a historical reference position representing the position of the historical image of the sign in the historical reference image based on the position of a boundary feature pixel at the light-dark boundary in the historical image of the sign.

[0064] Step 108, when the offset index value determined based on the target reference position and the historical reference position meets the preset offset condition, the edge map obtained by edge detection based on the sign image is matched with the edge template map preconfigured for the sign represented by the sign image to obtain a matching score.

[0065] The offset index value may be an offset angle value or an offset distance value. The offset angle may be a pixel offset angle or an actual offset angle. The pixel offset angle is an offset angle of a target reference position compared to a historical reference position. The actual offset angle value is an offset angle of a first position where the sign is located when an image collected for a slope area is obtained, compared to a second position where the sign is located when a historical reference image is obtained.

[0066] The offset distance can be a pixel offset distance or an actual offset distance. The pixel offset distance is the offset distance of the target reference position compared to the historical reference position. The actual offset distance value is the offset distance of the first position compared to the second position.

[0067] The preset offset condition is a condition corresponding to the indicator type of the offset indicator value and must be met to determine that the signboard may be offset. The indicator type includes the above-mentioned pixel offset angle, actual offset angle, pixel offset distance or actual offset distance. For example, if the indicator type is actual offset distance, the preset offset condition may be that the actual offset distance is greater than the preset distance. The preset distance is, for example, 10 cm.

[0068] Edge detection can be implemented using edge detection algorithms. Since light blocks and dark blocks are arranged alternately in the signboard, there is a light-dark boundary in the signboard image. The edge map obtained by edge detection based on the signboard image can include the extracted edge representing the light-dark boundary. Edge detection algorithms include the Canny algorithm (a multi-level edge detection algorithm developed by John F. Canny), the LoG algorithm (Laplacian of Gaussian, an algorithm that uses the Laplace operator to extract edges based on the Gaussian function) or others.

[0069] The edge template image is obtained based on a sign reference image of a sign represented by the sign image in a preset calibration image. The sign reference image can be obtained by detecting the preset calibration image using a trained sign detection model, or by manually annotating the preset calibration image.

[0070] Exemplarily, when the offset index value determined based on the target reference position and the historical reference position meets the preset offset condition, the camera may use the NCC algorithm (Normalized Cross-Correlation) to match the edge map obtained by edge detection based on the sign image based on the pre-configured edge template map of the sign represented by the sign image to obtain a matching score. The matching score may represent the degree of matching between the edge map corresponding to the sign image and the edge template map. The matching score may be the maximum matching score in the matching process. It can be understood that the higher the matching score, the higher the matching degree and the lower the possibility of the sign offset.

[0071] In one embodiment, the camera can determine the pixel offset vector of the target reference position relative to the historical reference position, determine the actual offset vector of the sign based on the pixel offset vector, determine the actual offset distance of the sign based on the actual offset vector, and when the actual offset distance is greater than the preset distance, determine that the offset index value meets the preset offset condition.

[0072] Wherein, the target reference position and the historical reference position can be represented by coordinates respectively, and the pixel offset vector can include the offset in the horizontal direction (x-axis direction) and the offset in the vertical direction (y-axis direction). For example, the target reference position can be (x1, y1), the historical reference position can be (x0, y0), and the pixel offset vector can be {Dx, Dy}, where Dx can be x1 minus x0, and Dy can be y1 minus y0. Since the size of the light block or dark block in the sign is known, and the size of the image area corresponding to the light block or dark block can be determined in the sign image, the mapping relationship between the image size of the sign image and the actual size of the sign can be determined, and the pixel offset vector can be mapped to the actual offset vector, and the modulus length of the actual offset vector can be calculated to obtain the actual offset distance of the sign.

[0073] In one embodiment, the camera can obtain each sign reference image representing the sign in the preset calibration image, and for each sign reference image, the width is reduced to the preset reference width, and the sign reference image is reduced, and then edge detection is performed to obtain an initial edge map, and the pixel values ​​of non-zero pixels that do not meet the boundary edge condition in the initial edge map are set to zero to obtain an edge template map corresponding to the sign represented by the sign reference image. The preset reference width is, for example, 48 pixels, that is, the width of the reduced sign reference image can be 48 pixels.

[0074] In one embodiment, the camera may traverse the non-zero pixels in the initial edge map, determine the horizontal gradient value and the vertical gradient value of the non-zero pixel for the traversed non-zero pixel, determine the gradient direction angle and the gradient amplitude of the non-zero pixel based on the horizontal gradient value and the vertical gradient value, and when the gradient direction angle is not within the horizontal representation angle range and is not within the vertical representation angle range, or the gradient amplitude is not greater than a preset intensity threshold, it is determined that the non-zero pixel does not meet the boundary edge condition.

[0075] Among them, the non-zero pixels in the initial edge map represent the pixels obtained as edges through edge detection. The Sobel operator (Sobel operator, including the convolution kernel in the horizontal direction and the convolution kernel in the vertical direction) is used to convolve the non-zero pixels to obtain the horizontal gradient value and the vertical gradient value of the non-zero pixel. The gradient direction angle can characterize the gradient direction of the non-zero pixel, which can be calculated using the inverse tangent function. The horizontal gradient value and the vertical gradient value are respectively recorded as Gx and Gy, and the gradient amplitude can be √(Gx*Gx+ Gy*Gy), where √ represents the square root of the value of (Gx*Gx+ Gy*Gy). The horizontal characterization angle range can be 0 degrees to 20 degrees, and the vertical characterization angle range can be 70 degrees to 110 degrees. The preset intensity threshold can be 80% of the maximum gradient amplitude in the initial edge map. For example, the maximum gradient amplitude can be 50, and the preset intensity threshold can be 40. When the gradient direction angle is within the horizontal representation angle range or within the vertical representation angle range, and the gradient amplitude is greater than the preset intensity threshold, it can be considered that the non-zero pixel meets the boundary edge condition, the non-zero pixel belongs to a horizontal or vertical strong edge, and no action is taken on the pixel value of the non-zero pixel.

[0076] In one embodiment, the step of performing edge detection on the sign image to obtain an edge map includes: performing edge detection on the sign image to obtain an edge map.

[0077] In one embodiment, the step of obtaining an edge map by performing edge detection based on a signboard image also includes: performing regional expansion of the area occupied by the signboard image in the image collected for the slope area according to a preset multiple, determining the expanded area, intercepting the expanded area from the image to obtain an expanded area image, and performing edge detection on the expanded area image after reducing it according to a preset reduction ratio to obtain an edge map corresponding to the signboard image.

[0078] The size of the expanded area is increased by a preset multiple compared to the size of the area occupied by the sign image in the image. For example, the preset multiple is recorded as k, the width and height of the area occupied by the sign image in the image are a and b respectively, then the width and height of the expanded area are a*(1+k) and b*(1+k) respectively. The preset multiple can be set according to engineering experience, for example, it can be 0.4 times. The preset reduction ratio can be the ratio value obtained by dividing the width of the sign reference image by the preset reference width.

[0079] The size of the edge map corresponding to the expanded area image may be larger than the size of the edge template map. When the NCC algorithm is used for matching, since the expanded area image is reduced according to a preset reduction ratio, a window consistent with the size of the edge template map may be set, and the window is used to perform window scanning processing in the edge map corresponding to the expanded area image, so as to find an image area similar to the edge template map in the edge map obtained from the expanded area image.

[0080] By performing regional expansion processing on the area occupied by the signboard image, even if the signboard is offset and the signboard image may not be fully recognized, the expanded area image can more accurately include the image presented by the signboard in the image, thereby improving the accuracy of subsequent matching with the edge template image.

[0081] Step 110 : when the matching score indicates that the sign represented by the sign image has shifted, a landslide detection result for the slope area is determined based on a shift start time when the sign represented by the sign image begins to shift.

[0082] The landslide detection result may include a detection result indicating that a landslide has occurred in the slope area, or a detection result indicating that a landslide has not occurred in the slope area. In the case where the landslide detection result indicates that a landslide has occurred in the slope area, a landslide warning message for the slope area may be issued. The landslide warning message may include, for example, "a landslide is suspected to have occurred in the slope area" and may carry the geographical location of the slope area. The landslide warning message may be pushed to a terminal near the geographical location or to a terminal of an operation and maintenance personnel. The terminal of the operation and maintenance personnel may run a program that can view the camera screen. The operation and maintenance personnel may view the real-time monitoring screen of the camera through the program and manually review the slope area to confirm whether a landslide has occurred in the slope area.

[0083] For example, when the matching score indicates that the sign represented by the sign image is offset, the camera can obtain the offset start time when the sign represented by the sign image begins to offset, and generate a detection result indicating that a landslide has occurred in the slope area when the duration from the offset start time to the acquisition time of the image acquired for the slope area satisfies the first duration condition. The first duration condition can be that the duration is greater than a first preset duration, such as 5 minutes, 10 minutes, 30 minutes or others.

[0084] In one embodiment, when the matching score is less than a first preset score threshold, the camera may determine that the matching score indicates that the sign represented by the sign image is offset. For example, the matching score may range from 0 to 1, and the first preset score threshold may be 0.5.

[0085] In one embodiment, when the matching score indicates that the sign represented by the sign image is offset, the camera may query the offset start time of the sign represented by the sign image, and when the offset start time of the sign is found, the offset start time is obtained; when the offset start time of the sign is not found, the acquisition time of the image captured for the slope area is used as the offset start time of the sign and stored. The offset start time may be queried from a preconfigured storage space. The storage space may be a local disk, a cloud storage space, etc. It is understandable that if the offset start time of the sign represented by the sign image is found, it means that the sign represented by the sign image has been offset before the acquisition time of the image captured for the slope area.

[0086] In one embodiment, when the matching score indicates that the sign represented by the sign image has not shifted, the camera may query the shift start time when the sign began to shift for the sign represented by the sign image, and when the shift start time of the sign is found, the found shift start time is deleted. Wherein, when the matching score is not less than the first preset score threshold, the camera may determine that the matching score indicates that the sign represented by the sign image has not shifted.

[0087] In the above landslide detection method, a signboard is set up in the slope area, and multiple signboard images can be identified from the images collected for the slope area. Since the light blocks and dark blocks in the signboard are arranged alternately, there are boundary feature pixels at the boundary between light and dark in the signboard image. Based on the position of the boundary feature pixels, the target reference position representing the position of the signboard image in the image can be determined. When the offset index value determined based on the target reference position and the historical reference position meets the preset offset condition, since the historical reference position is the signboard represented by the signboard image in the historical reference image, it can be preliminarily determined that the signboard may be offset. Furthermore, the edge map obtained by edge detection based on the signboard image is pre-assigned to the signboard represented by the signboard image. The edge template map is matched to obtain a matching score. The matching score can be used to accurately determine whether the signboard is offset. When it is determined through the matching score that the signboard is offset, the landslide detection result for the slope area is determined based on the offset start time when the signboard begins to offset. This can exclude the situation where the signboard accidentally offsets. In this way, since multiple levels of judgment are performed on the signboard, the landslide detection result obtained is more accurate. Moreover, by setting a signboard with light blocks and dark blocks arranged alternately in the slope area, collecting images of the slope area, and then performing a series of data processing to perform landslide detection, there is no need to lay multiple sensors and sensor support equipment in the slope area, which can reduce hardware resource consumption.

[0088] In an exemplary embodiment, after executing step 102, the camera may detect the position of the boundary feature pixel at the boundary between light and dark in the sign image among the multiple sign images, and when the position of the boundary feature pixel is not detected, perform the step of matching the edge map obtained by edge detection based on the sign image with the edge template map pre-configured for the sign represented by the sign image to obtain a matching score. Wherein, when the position of the boundary feature pixel is not detected, information representing that the sign is seriously offset may also be generated.

[0089] In an exemplary embodiment, the image and the historical reference image are collected by a camera located outside the slope area, and the signboard includes a close-up signboard or a distant signboard; the distance from the close-up signboard to the camera is closer than the distance from the distant signboard to the camera; step 110 also includes: when the time from the offset start time when the signboard represented by the signboard image begins to offset to the image acquisition time satisfies the first time condition, the type of the signboard represented by the signboard image is determined; when the type is a close-up signboard, a detection result representing a landslide in the slope area is generated.

[0090] Among them, the distance between the near-view sign and the camera can be within a first preset distance range, and the distance between the far-view sign and the camera can be within a second preset distance range, and the distance value in the second preset distance range is greater than the distance value in the first preset distance range. For example, the area covered by the camera's field of view may include a position in the slope within a distance of 10 meters to 300 meters from the camera, then the first preset distance range may be 10 meters to 150 meters, and the second preset distance range may be 170 meters to 300 meters. In the area where the distance to the camera is within the first preset distance range, a near-view sign can be set at intervals of a preset distance. In the area where the distance to the camera is within the second preset distance range, a far-view sign group can be set at intervals of a preset distance. The preset distance interval is, for example, 20 meters. A far-view sign group includes at least two far-view signs. When setting near-view signs or far-view signs, it is also necessary to consider that the images presented by different signs in the camera's picture do not block each other, so after setting the signs at the preset distance intervals, the position of the signs can also be fine-tuned according to the blocking situation.

[0091] The type of the sign can be detected while the sign detection model detects the target frame. The sign detection model can be configured to output the rectangular coordinate information of the target frame and the type of the sign. The type of the output sign includes one of a near-view sign and a far-view sign. The sign detection model can be trained based on multiple sample images and annotations for each sample image. The annotations for each sample image may include the rectangular coordinate information of the image mark of the sign in the sample image and the type to which it belongs.

[0092] The multiple sample images may be obtained by performing sample capacity expansion processing on the original images collected from the slope area under different lighting environments. The original image may include the images of multiple signboards in the slope area, and specifically may include the images of each near-view signboard and each distant view signboard. The sample capacity expansion processing may include at least one specific processing of occluding processing, cropping processing, grayscale transformation processing, adding Gaussian noise processing, and rotating processing of the local image area. The sample capacity expansion processing may be random, specifically, at least one specific processing may be randomly adopted, and the position or degree of processing during the specific processing may also be random, such as rotating a random angle.

[0093] In the present embodiment, since the distance from the foreground signboard to the camera is closer than the distance from the background signboard to the camera, when the signboard represented by the signboard image is a foreground signboard, the imaging of the signboard image is clear. Therefore, the previously determined offset index value and matching score are more accurate. Furthermore, when the duration from the offset start time when the signboard represented by the signboard image begins to offset to the image acquisition time satisfies the first duration condition, the landslide detection result for the slope area can be determined more accurately.

[0094] In an exemplary embodiment, the size of the light block or dark block in the distant sign is larger than the size of the light block or dark block in the near sign; the total number of light blocks and dark blocks in the distant sign is less than the total number of light blocks and dark blocks in the near sign; the distant sign in the slope area is divided into a plurality of distant sign groups, and different distant sign in each distant sign group meet the position proximity condition; step 110 also includes: when the type is a distant sign, determining the target sign in the distant sign group where the sign represented by the sign image is located; the duration from the offset start time of the target sign to the image acquisition time meets the first duration condition; when the number of target signs meets the preset number condition, generating a detection result representing the occurrence of landslide in the slope area.

[0095] The size of the light block or dark block in the distant sign is larger than the size of the light block or dark block in the near sign, so that even if the distant sign is far away from the camera, the boundary between the light block and the dark block in the distant sign is easier to display in the image. The total number of light blocks and dark blocks in the distant sign can be 4, of which there are 2 light blocks and 2 dark blocks. The total number of light blocks and dark blocks in the near sign can be 16, of which there are 4 light blocks and 4 dark blocks.

[0096] The position proximity condition is that the interval between two adjacent distant signboards in the same distant signboard group is within a preset signboard interval range. The preset signboard interval range is, for example, 0 to 0.5 meters. The preset quantity condition can be that the number of target signboards is greater than the preset number, or that the number of target signboards is greater than a second preset ratio of the total number of signboards in the distant signboard group. The preset number is, for example, 2, 3, or others. The second preset ratio is, for example, 50%. For example, the total number of signboards in the distant signboard group may be 3, and the preset quantity condition may be that the number of target signboards is greater than 1.5 (3 times 50%).

[0097] In this embodiment, since the distance between the distant signboard and the camera is relatively far, compared with the method of setting the same signboard at each position in the slope area, by setting the size of the light block or dark block in the distant signboard larger than the size of the light block or dark block in the near-view signboard, the boundary between the light block and the dark block in the distant signboard is easier to display in the image, thereby obtaining a more accurate offset index value and matching score; and, since the total number of light blocks and dark blocks in the distant signboard is less than the total number of light blocks and dark blocks in the near-view signboard, there are fewer light and dark boundaries in the signboard image corresponding to a distant signboard. Therefore, the distant signboards in the slope area are divided into multiple distant signboard groups, and different distant signboards in each distant signboard group meet the position proximity condition. The offset start time of the distant signs in the same group is jointly judged, which can further improve the accuracy of the landslide detection result.

[0098] In one embodiment, the signboard image among the multiple signboard images targeted by step 104 may be a target signboard image among the multiple signboard images. The camera may select a signboard image whose clarity meets the clarity condition from the signboard images representing the near-view signboard as the target signboard image; when the clarity of each of the signboard images representing the near-view signboard does not meet the clarity condition, different signboard images with the highest average clarity, which respectively represent different distant signboards in a distant signboard group, are selected from the signboard images representing the distant signboard as the target signboard image. The average clarity is the average clarity of each of the different signboard images representing different distant signboards in a distant signboard group.

[0099] In an exemplary embodiment, after step 108, the landslide detection method further includes: when the matching score indicates that the signboard represented by the signboard image is obstructed, obtaining the occlusion start time when the signboard represented by the signboard image begins to be obstructed; and when the duration from the occlusion start time to the image acquisition time satisfies a second duration condition, outputting occlusion alarm information indicating that the signboard represented by the signboard image is obstructed.

[0100] Among them, since the actual scene in the slope area may be more complicated, the signboard may be blocked or interfered with. For example, when the slope area is on the side of the road, it is easily affected by car lights and trees. Therefore, it is necessary to judge the obstruction of the signboard during the landslide detection process. The second duration condition can be that the duration is greater than the second preset duration, and the second preset duration is, for example, 5 minutes, 10 minutes, 30 minutes or other. The obstruction alarm information can indicate the identification of the signboard, such as the number and location of the signboard. The obstruction alarm information can be pushed to the terminal of the operation and maintenance personnel. The operation and maintenance personnel can view the real-time monitoring screen of the camera and manually review the signboard to confirm whether the signboard is blocked. When it is confirmed that the signboard is blocked, they can go to the slope area for on-site processing.

[0101] When the matching score is not less than the first preset score threshold and not greater than the second preset score threshold, it can be determined that the matching score indicates that the sign represented by the sign image is blocked. The second preset score threshold is greater than the first preset score threshold. For example, the first preset score threshold may be 0.5, and the second preset score threshold may be 0.7.

[0102] In the present embodiment, when it is determined through the matching score that the signboard represented by the signboard image is obstructed, further judgment is made through the start time of the signboard occlusion. When the duration from the occlusion start time to the image acquisition time satisfies the second duration condition, the situation where the signboard is accidentally obstructed can be ruled out. At this time, an occlusion alarm message indicating that the signboard represented by the signboard image is obstructed is output, and relevant operation and maintenance personnel can be notified to handle it, thereby improving the maintainability of the signboard.

[0103] In one embodiment, when the matching score indicates that the sign represented by the sign image is obstructed, the camera may query the occlusion start time of the sign represented by the sign image, and when the occlusion start time of the sign is found, the queried occlusion start time is obtained; when the occlusion start time of the sign is not found, the acquisition time of the image captured for the slope area is used as the occlusion start time of the sign being obstructed and stored. The occlusion start time may be queried from a preconfigured storage space. It is understandable that if the occlusion start time of the sign represented by the sign image is found, it means that the sign represented by the sign image was detected to be obstructed before the acquisition time of the image captured for the slope area.

[0104] In an exemplary embodiment, step 104 may include: for a sign image among multiple sign images, detecting the position of boundary feature pixels at the light and dark boundary of the sign image through a trained feature position detection model; when the number of boundary feature pixels is multiple, averaging the positions of the boundary feature pixels in the sign image to obtain a target reference position representing the position of the sign image in the image.

[0105] Among them, the feature position detection model can be obtained by training the target detection model to be trained. The target detection model is such as ‌YOLOv5s, Faster R-CNN or others. The output of the feature position detection model can be the coordinates of the boundary feature pixels. The feature position detection model can be trained based on multiple signboard sample images and annotations for each signboard sample image. Each signboard sample image can represent the signboard. The annotation of each signboard sample image may include the coordinates marked for the boundary feature pixels in the signboard sample image; for example, it may include the common pixels of the target light and dark boundary pairs in different light and dark boundaries in the signboard sample image, wherein the two target boundary edges represented by the target light and dark boundary pair intersect in the signboard represented by the signboard sample image, and the angle between the two target boundary edges is a right angle.

[0106] The multiple signboard sample images can be obtained by performing signboard sample capacity expansion processing on the original signboard images of the near-view signboard and the distant signboard under different lighting environments. The original signboard image can be intercepted from the original image collected from the slope area. The signboard sample capacity expansion processing can include at least one specific processing of local image area occlusion processing, area expansion processing, cropping processing, grayscale transformation processing, Gaussian noise addition processing, and rotation processing. The signboard sample capacity expansion processing can be random.

[0107] In the process of determining the historical reference position, the feature position detection model can be used to first determine the position of the boundary feature pixels at the light and dark boundary of the sign represented by the sign image in the historical image of the sign, and then based on the position of the boundary feature pixels at the light and dark boundary in the historical image of the sign, the historical reference position representing the position of the historical image of the sign in the historical reference image can be determined.

[0108] The position of the boundary feature pixel can be represented by coordinates. The position of the boundary feature pixels in the signboard image can be averaged by averaging the horizontal coordinates of multiple boundary feature pixels in the signboard image and averaging the vertical coordinates of the multiple boundary feature pixels. In this way, the horizontal and vertical coordinates of the target reference position can be obtained. That is, the target reference position can be the center of the positions of the multiple boundary feature pixels.

[0109] In this embodiment, the position of the boundary feature pixels in the light and dark boundary of the sign image can be quickly detected through the trained feature position detection model. Therefore, when the number of boundary feature pixels is multiple, the positions of the boundary feature pixels in the sign image are averaged to obtain the target reference position, which can more accurately characterize the position of the sign image in the image and create conditions for the subsequent determination of the offset index value.

[0110] In an exemplary embodiment, the degree of reflectivity of the bright blocks is greater than that of the dark blocks, and the image is collected from the slope area at the current time. Before step 102, the landslide detection method further includes the following steps: acquiring multiple historical images collected from the slope area in a historical period adjacent to the current time; determining a reference brightness based on the brightness of the image and the brightness of each of the multiple historical images; when the reference brightness meets a preset fill light condition, starting the fill light to illuminate each signboard set in the slope area, so that under the illumination of the fill light, the degree of reflectivity of the bright blocks in the same signboard is greater than that of the dark blocks.

[0111] The historical period before the current time may be a period of a preset duration from the current time. For example, the current time may be 10:00, the preset duration may be 1 minute, and the historical period may be one minute from 9:59. The multiple historical images may be a preset number of historical images, such as 5, 10 or other preset numbers. The reference brightness may be obtained by calculating the average brightness of the image captured at the current time and the brightness of each of the multiple historical images.

[0112] The preset fill light condition may be that the reference brightness is less than a preset nighttime brightness threshold. Meeting the preset fill light condition may indicate that it is currently at night. The fill light may be integrated into the camera. The fill light may also be independent of the camera, and whether it is started may be controlled by the camera or the server. By setting the reflective properties of the first material for making or spraying the light-colored blocks to be better than the reflective properties of the second material for making or spraying the dark-colored blocks, the reflective degree of the light-colored blocks may be greater than that of the dark-colored blocks; or, the first material for making or spraying the light-colored blocks may be reflective material, while the second material for making or spraying the dark-colored blocks may be non-reflective material, so that the reflective degree of the light-colored blocks may be greater than that of the dark-colored blocks.

[0113] In this embodiment, when the reference brightness determined based on the brightness of the image and the brightness of each of the multiple historical images meets the preset fill-light condition, the fill-light is started to illuminate the various signboards set in the slope area. In this way, in the case of insufficient light, such as at night, fill-light can be provided in time. Moreover, because the degree of reflection of the light blocks in the same signboard is greater than that of the dark blocks under the illumination of the fill-light, the light blocks and dark blocks in the same signboard still have obvious light-dark boundaries after imaging, so that the position of the boundary feature pixel can be accurately determined, creating conditions for obtaining accurate landslide detection results, thereby achieving all-weather landslide detection.

[0114] In one embodiment, the above-mentioned landslide detection method also includes the following steps: the fill light can be integrated into a light sensing module, the light sensing module is independent of the camera and the server, and a photosensitive sensor can be provided in the light sensing module; the light sensing module can determine whether it is currently at night through the photosensitive sensor, and automatically start the fill light if it is determined to be at night.

[0115] In a specific application scenario, the slope area can be a part of a hillside, which can be distributed on one side of a highway. An intelligent recognition camera is set up on the roadside in front of the hillside on one side of the highway where there is no risk of collapse. A telephoto lens is used to cover the slope area formed along the foot of the hillside adjacent to the highway toward the hillside away from the highway. Each position of the slope area is within 10 to 300 meters from the camera. Make near-view signboards and far-view signboards. Specifically, the light blocks and dark blocks in the signboards are both square in shape and of the same size. See, for example, Figure 2 The foreground signboard sample schematic diagram shown in FIG. 1 may include 4*4 color blocks, a total of 16 color blocks, arranged in such a way that one of every two adjacent color blocks is a light color block and the other is a dark color block, see FIG. Figure 3 The example of the distant view sign is shown in the schematic diagram. The distant view sign may include 2*2 color blocks, a total of 4 color blocks, arranged in such a way that one of every two adjacent color blocks is a light block and the other is a dark block. In the slope area, from 10 meters away from the camera to 150 meters away from the camera, a near view sign is set every 20 meters, and from 170 meters away from the camera to 300 meters away from the camera, a distant view sign group is set every 20 meters. Each distant view sign group includes 3 distant view signs, and the images presented by different signs in the camera's picture do not block each other. The distribution of signs and cameras can be seen in the following example. Figure 4 Schematic diagram of landslide detection scenario shown.

[0116] The intelligent recognition camera may include a fill light control module, a sign detection module, an offset calculation module based on boundary feature pixels, an offset and occlusion detection module based on matching, a display module, and a real-time detection control module. Figure 2 , Figure 3 , Figure 4 As well as the camera modules, see Figure 5 As shown in the schematic flow chart of landslide detection steps, the above landslide detection method may specifically include the following steps.

[0117] In completion Figure 4 After the construction is placed as shown, the configuration phase is entered. During the configuration process, the sign detection model and the feature position detection model can be obtained through camera and / or server training. The offset calculation module based on the boundary feature pixels can load the pre-trained sign detection model through the NPU (Neural Processing Unit, neural network processor). The offset calculation module based on the boundary feature pixels can load the pre-trained feature position detection model through the NPU. Then, the camera collects images of the slope area to obtain calibration images, and the calibration images are detected by the sign detection model to obtain the sign reference image obtained by the preliminary detection. The display module of the camera can display the sign reference image obtained by the preliminary detection to the application running on the terminal of the operation and maintenance personnel (it can be on the WEB side, that is, the web page side, or on the APP side). After checking, the operation and maintenance personnel confirm whether the sign reference image corresponding to each sign is missed. If there is a missed detection, it can be manually marked and supplemented, and then each sign reference image representing the sign is stored in the configuration file.

[0118] Further, the camera and / or the server complete the initialization step of the offset and occlusion detection module based on matching. The initialization step may include: obtaining each signboard reference image representing the signboard in the preset calibration image, reducing the width of each signboard reference image to 48 pixels, reducing the signboard reference image, and performing edge detection using the Canny algorithm to obtain an initial edge map; traversing the non-zero pixels in the initial edge map, performing convolution with the Sobel operator for the traversed non-zero pixels, determining the horizontal gradient value and vertical gradient value of the non-zero pixel, determining the gradient direction angle and gradient amplitude of the non-zero pixel based on the horizontal gradient value and the vertical gradient value, when the gradient direction angle is within the horizontal representation angle range or within the vertical representation angle range, and the gradient amplitude is greater than the preset intensity threshold, it is determined that the non-zero pixel meets the boundary edge condition, otherwise it is determined that the non-zero pixel does not meet the boundary edge condition, and the pixel values ​​of the non-zero pixels that do not meet the boundary edge condition in the initial edge map are set to zero, and the edge template map corresponding to the signboard represented by the signboard reference image is obtained.

[0119] After the configuration is completed, the real-time detection stage begins. The camera is started to record video in real time. At the first startup, the reference image of each sign representing the sign is obtained from the configuration file, and the real-time detection control module of the camera obtains a video frame of the current time at a preset time interval as the image collected for the slope area.

[0120] The real-time detection control module of the camera can instruct the sign detection module to input the image collected for the slope area into the sign detection model, obtain the multiple target frames detected in the image output by the sign detection model, and respectively capture the image blocks identified by the multiple target frames in the image as sign images, so as to identify the multiple sign images in the image collected for the slope area.

[0121] The real-time detection control module of the camera can instruct the offset calculation module based on the boundary feature pixels to detect the position of the boundary feature pixels in the light and dark boundary of the sign image in the multiple sign images through the trained feature position detection model. When the number of boundary feature pixels is multiple, the positions of the boundary feature pixels in the sign image are averaged to obtain the target reference position representing the position of the sign image in the image. Figure 3 , adjacent light blocks and dark blocks can form a dividing edge, Figure 3 There are 4 dividing edges in total, and the two dividing edges with right angles can be Figure 3 The two target boundary edges represented by the target light and dark boundary pair in the sign image corresponding to the sign shown in the figure, the intersection of the two target boundary edges in the sign image can be the boundary feature pixel. Figure 3 The intersection points of the four dividing edges are the same. Figure 3 There may be a boundary feature pixel in the signboard image corresponding to the signboard shown.

[0122] The historical reference image can be a preset calibration image. The real-time detection control module of the camera can obtain the signboard reference image of the signboard represented by the signboard image in the preset calibration image from the configuration file, instruct the offset calculation module based on the boundary feature pixels to detect the position of the boundary feature pixels in the light and dark boundary of the signboard reference image through the trained feature position detection model, and determine the initial reference position representing the position of the signboard reference image in the calibration image based on the position of the boundary feature pixels at the light and dark boundary in the signboard reference image, and use the initial reference position as the historical reference position of the signboard represented by the signboard image in the historical reference image.

[0123] The real-time detection control module of the camera can instruct the matching-based offset and occlusion detection module to match the edge map obtained by edge detection based on the signboard image with the pre-configured edge template map of the signboard represented by the signboard image to obtain a matching score when the offset index value determined based on the target reference position and the historical reference position meets the preset offset condition.

[0124] When the matching score indicates that the signboard represented by the signboard image has shifted, the real-time detection control module of the camera can obtain the shift start time when the signboard begins to shift; when the duration from the shift start time when the signboard represented by the signboard image begins to shift to the image acquisition time meets the first duration condition, the type of the signboard represented by the signboard image is determined; when the type is a close-up signboard, a detection result representing a landslide in the slope area is generated, instructing the display module to push landslide alarm information for the slope area to the application in the terminal of the operation and maintenance personnel.

[0125] When the type is a distant signboard, the target signboard is determined in the distant signboard group where the signboard represented by the signboard image is located; the duration from the offset start time of the target signboard to the image acquisition time satisfies the first duration condition; when the number of target signboards meets the preset number condition, a detection result representing the occurrence of a landslide in the slope area is generated, and the display module is instructed to push landslide warning information for the slope area to the application in the terminal of the operation and maintenance personnel.

[0126] When the matching score indicates that the signboard represented by the signboard image is obstructed, the occlusion start time when the signboard represented by the signboard image begins to be obstructed is obtained; when the duration from the occlusion start time to the image acquisition time satisfies the second duration condition, occlusion alarm information indicating that the signboard represented by the signboard image is obstructed is output, and the display module is instructed to push the occlusion alarm information for the signboard to the application in the terminal of the operation and maintenance personnel.

[0127] When the matching score indicates that the sign represented by the sign image has not shifted, the shift start time at which the sign represented by the sign image starts to shift is deleted.

[0128] The fill light control module of the camera can obtain multiple historical images collected on the slope area in the adjacent historical period before the current time during the real-time detection process; determine the reference brightness based on the brightness of the image and the brightness of each of the multiple historical images; when the reference brightness meets the preset fill light condition, start the fill light to illuminate each signboard set in the slope area, so that under the illumination of the fill light, the reflectivity of the bright blocks in the same signboard is greater than that of the dark blocks.

[0129] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a landslide detection device for implementing the landslide detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more landslide detection device embodiments provided below can refer to the limitations on the landslide detection method above, and will not be repeated here.

[0131] In an exemplary embodiment, Figure 6 As shown, a landslide detection device 600 is provided, comprising: an identification module 610 and a detection module 620, wherein:

[0132] The recognition module 610 is used to recognize multiple signboard images in the image collected for the slope area; the signboard image represents the signboard set in the slope area, and the light blocks and dark blocks in the signboard are arranged alternately.

[0133] Detection module 620 is used to determine, for a signboard image among multiple signboard images, a target reference position representing the position of the signboard image in the image based on the position of the boundary feature pixels at the light-dark boundary in the signboard image; obtain the historical reference position of the signboard represented by the signboard image in the historical reference image; when the offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, match the edge map obtained by edge detection based on the signboard image with the edge template map preconfigured for the signboard represented by the signboard image to obtain a matching score; when the matching score indicates that the signboard represented by the signboard image has shifted, determine the landslide detection result for the slope area based on the shift start time when the signboard represented by the signboard image begins to shift.

[0134] In an exemplary embodiment, the image and the historical reference image are collected by a camera located outside the slope area, and the signboard includes a close-up signboard or a distant signboard; the distance from the close-up signboard to the camera is closer than the distance from the distant signboard to the camera; the detection module 620 is also used to determine the type of the signboard represented by the signboard image when the duration from the offset start time of the signboard represented by the signboard image to the image acquisition time meets the first duration condition; when the type is a close-up signboard, a detection result representing a landslide in the slope area is generated.

[0135] In an exemplary embodiment, the size of the light block or dark block in the distant sign is larger than the size of the light block or dark block in the near sign; the total number of light blocks and dark blocks in the distant sign is less than the total number of light blocks and dark blocks in the near sign; the distant signboards in the slope area are divided into multiple distant signboard groups, and different distant signboards in each distant signboard group meet the position proximity condition; the detection module 620 is also used to determine the target signboard in the distant signboard group where the signboard represented by the signboard image is located when the type is a distant signboard; the time duration from the offset start time of the target signboard to the image acquisition time satisfies the first time duration condition; when the number of target signboards meets the preset number condition, a detection result representing the occurrence of landslide in the slope area is generated.

[0136] In an exemplary embodiment, the detection module 620 is also used to obtain the occlusion start time when the sign represented by the sign image begins to be occluded when the matching score indicates that the sign represented by the sign image is occluded; and when the duration from the occlusion start time to the image acquisition time satisfies a second duration condition, output occlusion alarm information indicating that the sign represented by the sign image is occluded.

[0137] In an exemplary embodiment, the detection module 620 is also used to detect the position of boundary feature pixels at the light and dark boundary of a sign image among multiple sign images through a trained feature position detection model; when the number of boundary feature pixels is multiple, the positions of the boundary feature pixels in the sign image are averaged to obtain a target reference position representing the position of the sign image in the image.

[0138] In an exemplary embodiment, the landslide detection device 600 also includes a fill light control module, which is used to obtain multiple historical images collected from the slope area in a historical period adjacent to the current time; determine a reference brightness based on the brightness of the image and the brightness of each of the multiple historical images; when the reference brightness meets the preset fill light condition, start the fill light to illuminate each signboard set in the slope area, so that under the illumination of the fill light, the reflectivity of the bright blocks in the same signboard is greater than that of the dark blocks.

[0139] Each module in the above landslide detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0140] In an exemplary embodiment, a computer device is provided. The computer device may be a camera, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an image acquisition unit, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory, the image acquisition unit and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The image acquisition unit of the computer device is used to acquire images. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data to be stored when executing a landslide detection method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a landslide detection method is implemented.

[0141] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0144] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0145] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0147] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A landslide detection method, characterized in that: The method comprises: Identify multiple signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged; For a signboard image among the plurality of signboard images, determining a target reference position representing a position of the signboard image in the image based on a position of a boundary feature pixel at a light-dark boundary in the signboard image; Obtaining a historical reference position of the sign represented by the sign image in the historical reference image; In the case where the offset index value determined based on the target reference position and the historical reference position meets a preset offset condition, an edge map obtained by edge detection based on the sign image is matched with an edge template map pre-configured for the sign represented by the sign image to obtain a matching score; When the matching score indicates that the sign represented by the sign image has shifted, a landslide detection result for the slope area is determined based on a shift start time when the sign represented by the sign image starts to shift.

2. The method according to claim 1, characterized in that The image and the historical reference image are collected by a camera located outside the slope area, and the signboard includes a near-view signboard or a far-view signboard; the distance between the near-view signboard and the camera is closer than the distance between the far-view signboard and the camera; The determining of the landslide detection result for the slope area based on the starting time of the signboard shifting represented by the signboard image comprises: When the time from the start time of the deviation of the sign represented by the sign image to the acquisition time of the image meets the first time condition, the type of the sign represented by the sign image is determined; When the type is a close-up signboard, a detection result indicating that a landslide has occurred in the slope area is generated.

3. The method according to claim 2, characterized in that The size of the light block or dark block in the distant view sign is larger than the size of the light block or dark block in the near view sign; the total number of light blocks and dark blocks in the distant view sign is less than the total number of light blocks and dark blocks in the near view sign; the distant view sign in the slope area is divided into a plurality of distant view sign groups, and different distant view sign in each distant view sign group meet the position proximity condition; The determining of the landslide detection result for the slope area based on the starting time of the signboard shifting represented by the signboard image comprises: When the type is a distant sign, a target sign is determined in the distant sign group where the sign represented by the sign image is located; the duration from the offset start time when the target sign starts to offset to the acquisition time of the image satisfies the first duration condition; When the number of the target signboards meets a preset number condition, a detection result indicating that a landslide has occurred in the slope area is generated.

4. The method according to claim 1, characterized in that: The method further comprises: When the matching score indicates that the sign represented by the sign image is blocked, obtaining the blocking start time when the sign represented by the sign image starts to be blocked; When the duration from the occlusion start time to the image acquisition time satisfies a second duration condition, occlusion warning information indicating that the sign represented by the sign image is occluded is output.

5. The method according to claim 1, characterized in that The step of determining, for a sign image among the plurality of sign images, a target reference position representing a position of the sign image in the image based on a position of a boundary feature pixel at a light-dark boundary in the sign image, comprises: For a signboard image among the plurality of signboard images, detecting the position of a boundary feature pixel at a light-dark boundary of the signboard image by using a trained feature position detection model; When the number of the boundary feature pixels is multiple, the positions of the boundary feature pixels in the sign image are averaged to obtain a target reference position representing the position of the sign image in the image.

6. The method according to any one of claims 1 to 5, characterized in that: The light reflectivity of the bright color block is greater than that of the dark color block, the image is collected from the slope area at the current time, and the method further includes: Acquire a plurality of historical images collected on the slope area in a historical period adjacent to the current time; Determining a reference brightness based on the brightness of the image and the brightness of each of the plurality of historical images; When the reference brightness meets the preset fill light condition, the fill light is started to illuminate each signboard set in the slope area, so that under the illumination of the fill light, the reflectivity of the bright blocks in the same signboard is greater than that of the dark blocks.

7. A landslide detection device, characterized in that: The device comprises: A recognition module, used to recognize a plurality of signboard images in an image collected for a slope area; the signboard images represent signboards set in the slope area, and light blocks and dark blocks in the signboards are alternately arranged; A detection module is used to determine, for a signboard image among the multiple signboard images, a target reference position representing the position of the signboard image in the image based on the position of a boundary feature pixel at a light-dark boundary in the signboard image; obtain a historical reference position of the signboard represented by the signboard image in a historical reference image; and, when an offset index value determined based on the target reference position and the historical reference position satisfies a preset offset condition, match an edge map obtained by edge detection based on the signboard image with an edge template map preconfigured for the signboard represented by the signboard image to obtain a matching score; when the matching score indicates that the signboard represented by the signboard image has shifted, determine a landslide detection result for the slope area based on a shift start time when the signboard represented by the signboard image begins to shift.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images