Elevator shaft pit foreign matter detection method and device, electronic equipment and storage medium

Abstract

The invention discloses an elevator shaft pit foreign matter detection method and device, electronic equipment and a storage medium, and the method comprises the steps: extracting a ground region image from an original image of a shaft pit region, and determining a candidate target according to the ground region image, and inputting the local peak map of the candidate target into the target classification model to obtain a classification result of the candidate target, and determining that the foreign matter exists in the shaft pit when the classification result is the foreign matter, so that three-dimensional scene reconstruction of the shaft pit after image acquisition and recognition of the position and height of the object after world coordinate system transformation are not needed. The calculation amount of foreign matter recognition is reduced, the method is suitable for recognizing foreign matter in hardware equipment with low calculation power, the real-time performance of foreign matter detection is improved, the scene that the requirement for controlling the real-time performance of an elevator is high according to the foreign matter recognition result is met, in addition, foreign matter recognition is conducted on a local peak graph through the target classification model, height calculation is not needed, and the method is convenient to implement. The method is suitable for the condition that the relation between the foreign matters in the shaft pit is complex, and the foreign matter recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, electronic equipment and storage medium for detecting foreign objects in a bottom pit of an elevator shaft. Background Art

[0002] The elevator moves up and down in the hoistway, and the hoistway pit environment is crucial to the safety of the elevator. During the operation of the elevator, if people or foreign objects accidentally enter the hoistway, it will cause serious personnel safety accidents and damage the structure of the elevator, affecting the normal operation of the elevator. Therefore, it is necessary to detect foreign objects in the hoistway pit.

[0003] In the prior art, to detect foreign objects in a shaft pit, it is necessary to first reconstruct the three-dimensional scene of the shaft pit, project the reconstructed three-dimensional scene data into a world coordinate system, and determine the position and height of objects in the shaft pit based on the data in the world coordinate system. The three-dimensional scene reconstruction and coordinate transformation require large amounts of computation, and the real-time performance of foreign object detection is poor in hardware devices with low computing power. In addition, there are deviations in the detection results when the relationship between foreign object targets in the shaft pit is complex (for example, one target partially covers another target). Summary of the invention

[0004] The present invention provides a method, device, electronic equipment and storage medium for detecting foreign objects in an elevator shaft pit, so as to solve the problems of large calculation amount, poor real-time performance and deviation in detection results in the existing shaft pit foreign object detection.

[0005] In a first aspect, the present invention provides a method for detecting foreign matter in an elevator shaft pit, comprising:

[0006] Obtaining the original image of the shaft pit area;

[0007] Extracting a ground area image from the original image;

[0008] Determine a candidate target according to the ground area image, and obtain a local peak map of the candidate target;

[0009] Inputting the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target;

[0010] When the classification result is foreign matter, it is determined that there is foreign matter in the shaft bottom pit.

[0011] Optionally, extracting a ground area image from the original image includes:

[0012] Setting a cue point in the original image;

[0013] The original image after setting the cue point is input into the pre-trained SAM model to obtain multiple sub-regions;

[0014] Calculate the comprehensive score of each sub-area, and determine the sub-area with the largest comprehensive score as the ground area;

[0015] The ground area is intercepted from the original image to obtain a ground area image.

[0016] Optionally, calculate a comprehensive score for each sub-area, including:

[0017] Determine the minimum bounding rectangle of each sub-region, and calculate the ratio of the long side to the short side of the minimum bounding rectangle to obtain the aspect ratio;

[0018] Determine, from among the sub-regions, a sub-region whose aspect ratio is less than a preset threshold to obtain a candidate sub-region;

[0019] Calculating the area of ​​each candidate sub-region and the distance from the centroid of each candidate sub-region to a preset reference point;

[0020] The product of a preset coefficient and the distance is calculated, and the difference between the area and the product is calculated to obtain a comprehensive score of the candidate sub-region.

[0021] Optionally, determining a candidate target according to the ground area image and obtaining a local peak map of the candidate target includes:

[0022] Performing filtering processing on the ground area image to obtain a filtered ground area image;

[0023] A preset gradient algorithm is used to calculate the gradient of the filtered ground area image to obtain a gradient image;

[0024] Performing local peak filtering on the gradient image to obtain a local peak map;

[0025] A binary image of a local peak image subjected to binarization processing based on a pre-configured threshold;

[0026] A candidate target is determined based on the binary image, and a local peak map of the candidate target is intercepted from the local peak map.

[0027] Optionally, performing local peak filtering on the gradient image to obtain a local peak map includes:

[0028] The local peak image is obtained by performing local peak filtering on the gradient image in the following manner:

[0029] L = D - a × f (D);

[0030] Among them, L is the local peak image, D is the gradient image, f is the filter function, and a is the coefficient.

[0031] Optionally, determining a candidate target based on the binary image includes:

[0032] Performing boundary enhancement processing on the binary image, and performing contour detection on the binary image after the boundary enhancement processing to obtain first contours of multiple targets;

[0033] Counting the number of pixels of the plurality of first contours, and filtering the first contours whose number of pixels is less than a preset number, to obtain a second contour;

[0034] Determine a minimum circumscribed rectangle of the second contour, and calculate the area of ​​the minimum circumscribed rectangle of the second contour;

[0035] A second contour whose minimum circumscribed rectangle area is greater than a preset value is determined as a candidate contour, and an object corresponding to the candidate contour in the binary image is determined as a candidate object.

[0036] Optionally, the target classification model is trained by the following steps:

[0037] Acquire a training data set, wherein the training data set includes local peak map samples of each target sample and a first category of the target sample;

[0038] Constructing a target classification model and initializing the target classification model;

[0039] Randomly extracting a local peak map sample of the target sample and inputting it into the target classification model to obtain a second category of the target sample;

[0040] calculating a loss value using the first category and the second category;

[0041] Determine whether the preset training conditions are met;

[0042] If so, determining that the target classification model has completed training;

[0043] If not, the model parameters of the target classification model are adjusted according to the loss value, and the step of randomly extracting local peak map samples of the target samples and inputting them into the target classification model is returned.

[0044] In a second aspect, the present invention provides a device for detecting foreign matter in a pit of an elevator shaft, comprising:

[0045] The original image acquisition module is used to acquire the original image of the pit area of ​​the shaft;

[0046] A ground area image extraction module, used to extract a ground area image from the original image;

[0047] A local peak image acquisition module, used to determine a candidate target according to the ground area image and acquire a local peak image of the candidate target;

[0048] A classification module, used for inputting the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target;

[0049] A foreign matter determination module is used to determine that there is a foreign matter in the bottom pit of the shaft when the classification result is a foreign matter.

[0050] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0051] at least one processor; and

[0052] a memory communicatively connected to the at least one processor; wherein,

[0053] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the elevator shaft pit foreign object detection method described in any one of the first aspects of the present invention.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the elevator shaft pit foreign object detection method as described in any one of the first aspects of the present invention when executed.

[0055] The embodiment of the present invention extracts a ground area image from an original image of a shaft pit area, and after determining a candidate target from the ground area image, further obtains a local peak map of the candidate target, and inputs the local peak map into a target classification model to obtain a classification result. When the classification result is a foreign object, it is determined that a foreign object exists in the shaft pit. There is no need to collect images and then reconstruct a three-dimensional scene of the shaft pit and identify the position and height of the object after performing a world coordinate system transformation. This reduces the amount of calculation for foreign object recognition, is suitable for identifying foreign objects in hardware devices with low computing power, improves the real-time performance of foreign object detection, and meets the scenario where the real-time performance requirements for controlling an elevator are high according to the foreign object recognition results. In addition, foreign objects are identified through a local peak map using a target classification model without calculating the height. This is suitable for situations where the relationship between foreign objects in the shaft pit is complex, and improves the accuracy of foreign object recognition.

[0056] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 is a flow chart of a method for detecting foreign objects in an elevator shaft pit provided in Embodiment 1 of the present invention;

[0059] Figure 2 is a flow chart of a method for detecting foreign matter in an elevator shaft pit provided in Embodiment 2 of the present invention;

[0060] Figure 3 This is a schematic diagram of adding cue points to the original image;

[0061] Figure 4 It is a schematic diagram of determining a ground area from a plurality of areas and extracting a ground area image from an original image;

[0062] Figure 5 It is a schematic diagram of determining candidate targets based on a binary image;

[0063] Figure 6 It is a schematic diagram of the training data annotation;

[0064] Figure 7 It is a schematic diagram of the classification results obtained by the target classification model for candidate target identification;

[0065] Figure 8 It is a structural schematic diagram of a foreign body detection device for an elevator shaft pit provided in Embodiment 3 of the present invention;

[0066] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0067] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0068] Embodiment 1

[0069] Figure 1This is a flow chart of a method for detecting foreign objects in an elevator shaft pit provided in Embodiment 1 of the present invention. This embodiment can be applied to identifying whether there are foreign objects on the floor of an elevator shaft pit. This method can be performed by an elevator shaft pit foreign object detection device. The elevator shaft pit foreign object detection device can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the elevator shaft pit foreign body detection method comprises:

[0070] S101, obtaining an original image of the shaft bottom pit area.

[0071] The shaft of this embodiment may be the shaft in which a vertical elevator (vertical lift) operates, and the shaft pit area may be the area of ​​the shaft close to the bottom plate. In one embodiment, a camera may be installed on the side wall of the shaft close to the pit, and an image of the shaft pit area is captured by the camera to obtain an original image, which includes images of the ground of the shaft pit, the side walls of the shaft, and other areas, wherein the original image may be an RGB image.

[0072] S102: extracting a ground area image from the original image.

[0073] In this embodiment, the ground area image may refer to an image of the ground area in the shaft pit. In one embodiment, the original image may be input into a semantic segmentation model to obtain the ground area in the original image, and the ground area may be cut out from the original image to obtain the ground area image.

[0074] In another embodiment, after the camera is fixed, a preset area of ​​the ground can be delineated in the image based on the relative position of the camera and the ground, the focal length of the camera and other optical parameters, and the image of the preset area can be captured from the original image to obtain the ground area image. This embodiment does not limit the method of extracting the ground area image from the original image.

[0075] S103: Determine candidate targets according to the ground area image, and obtain a local peak map of the candidate targets.

[0076] In this embodiment, the candidate target may refer to an object in the ground pit detected based on the ground area image. The object may include elevator equipment installed on the ground area in the shaft pit, and may also include foreign objects other than elevator equipment that fall into the shaft, such as people or other objects.

[0077] In one embodiment, the ground area image can be first filtered to obtain a filtered ground area image after noise is removed, and then the gradient of the filtered ground area image is calculated to obtain a gradient image, and the gradient image is further peak filtered to obtain a local peak map, and the local peak map is binarized to obtain a binary image. After contour detection is performed on the binary image, candidate targets are screened out through the detected contours, and the local peak map of the candidate targets is cut out from the local peak map.

[0078] S104: Input the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target.

[0079] The target classification model can be a convolutional neural network (CNN), a recursive neural network (RNN), a deep neural network (DNN), etc. After training, the target classification model can identify the type of the target when a local peak map of a target is input to obtain a classification result. The classification result may include a classification result of a foreign object or a non-foreign object. Specifically in this embodiment, after determining the candidate target and obtaining the local peak map of the candidate target, the local peak map of the candidate target can be input into the target classification model to obtain the classification result of the candidate target.

[0080] S105. When the classification result is foreign matter, it is determined that there is foreign matter in the bottom pit of the shaft.

[0081] If the classification result of the candidate target is a foreign object, the candidate target is determined to be a foreign object, and the information of the candidate target can be output. For example, the candidate target detection frame can be output, such as the position and size of the detection frame, so as to determine whether the elevator operation is affected according to the position and size of the candidate target whose classification result is a foreign object, and control the elevator operation. For example, when the foreign object affects the elevator, the elevator can be controlled to stop running, and the elevator operation can be controlled again after the foreign object is cleared.

[0082] This embodiment extracts a ground area image from the original image of the shaft pit area, and after determining the candidate target from the ground area image, further obtains a local peak map of the candidate target, inputs the local peak map into the target classification model to obtain a classification result, and determines that there is a foreign object in the shaft pit when the classification result is a foreign object. There is no need to collect images and then reconstruct the three-dimensional scene of the shaft pit and identify the position and height of the object after transforming the world coordinate system, which reduces the calculation amount of foreign object recognition and is suitable for identifying foreign objects in hardware devices with low computing power, improves the real-time performance of foreign object detection, and meets the scene with high real-time requirements for controlling the elevator according to the foreign object recognition results. In addition, foreign objects are identified by the local peak map through the target classification model without calculating the height, which is suitable for situations where the relationship between foreign objects in the shaft pit is complex, and improves the accuracy of foreign object recognition.

[0083] Embodiment 2

[0084] Figure 2 This is a flow chart of a method for detecting foreign objects in an elevator shaft pit provided in Embodiment 2 of the present invention. Embodiment 2 of the present invention is optimized based on Embodiment 1 above. Figure 2 As shown, the elevator shaft pit foreign body detection method comprises:

[0085] S201, obtaining an original image of the shaft bottom pit area.

[0086] In this embodiment, a camera can be installed on the side wall of the shaft close to the pit. During the operation of the elevator, the camera is used to collect original images of the shaft pit area. The original images include images of the ground of the shaft pit, the side walls of the shaft and other areas.

[0087] S202, setting cue points in the original image, and inputting the original image after the cue points are set into a pre-trained SAM model to obtain a plurality of sub-regions.

[0088] The SAM model (Segment Anything Model) is a large artificial intelligence model based on machine vision. It is an image segmentation model based on prompt points. The SAM model training and image segmentation principles can refer to the relevant technologies of the existing SAM model and will not be described in detail here.

[0089] like Figure 3 As shown, in Figure 3 In the figure a, the original image of the shaft pit captured by the camera is shown. Figure 3 Figure b is the original image after adding hint points (multiple white points in b).

[0090] In this embodiment, after the original image with the cue points added is input into the SAM model, the original image is segmented into multiple sub-regions by the SAM model, that is, the SAM model outputs a segmented image, and the segmented image includes multiple sub-regions, such as Figure 4 The segmented image shown in c includes multiple sub-regions.

[0091] S203: Calculate the comprehensive score of each sub-area, and determine the sub-area with the largest comprehensive score as the ground area.

[0092] In one embodiment, the minimum enclosing rectangle of each sub-region can be determined, and the ratio of the long side to the short side of the minimum enclosing rectangle can be calculated to obtain the aspect ratio. From each sub-region, a sub-region whose aspect ratio is less than a preset threshold is determined to obtain a candidate sub-region. For each candidate sub-region, the area of ​​the candidate sub-region is calculated, as well as the distance from the center of gravity of each candidate sub-region to a preset reference point, the product of the preset coefficient and the distance is calculated, and the difference between the area and the product is calculated to obtain a comprehensive score of the candidate sub-region.

[0093] In this embodiment, the preset reference point may be a point preset in the original image. For example, the preset reference point may be the intersection of the center line in the x-axis direction of the original image (the position of 50% in the x-axis direction) and the dividing line of 75% (or other proportions) from top to bottom in the y-axis direction (e.g. Figure 3 The point P0 shown in c is the preset reference point), that is, the preset reference point can be a point centered in the lower part of the original image (that is, a point close to the ground area). Of course, the preset reference point can also be a point at other positions. This embodiment does not limit the setting method of the preset reference point.

[0094] For each sub-region, the number of pixels in the sub-region can be calculated as the area of ​​the sub-region. The center of gravity of the sub-region can be the first-order moment of the sub-region, that is, the coordinates of each pixel in the sub-region are multiplied by the gray value, and the sum of the products is calculated by the ratio of the area to obtain the center of gravity position. The comprehensive score of the sub-region is further calculated by the following formula:

[0095] Score = S - a × d;

[0096] Where s is the area of ​​the sub-region, a is the coefficient, and d is the distance from the center of gravity of the sub-region to the preset reference point (the distance can be expressed as the number of pixels between two points). After obtaining the comprehensive score of each sub-region, the sub-region with the largest comprehensive score is determined as the ground region, such as Figure 4 The diagram in (d) is a schematic diagram of the determined ground area. According to the calculation formula of the comprehensive score Score, Figure 4 The score of the ground area is the largest, so the ground area can be accurately determined.

[0097] S204: intercepting the ground area from the original image to obtain a ground area image.

[0098] Specifically, Figure 4 The middle e shows Figure 3 Schematic diagram of the ground area image captured from the original image in a.

[0099] S205 , performing filtering processing on the ground area image to obtain a filtered ground area image.

[0100] In this embodiment, the filtering process can be Gaussian filtering, median filtering, mean filtering and the like, that is, the pixel value of each pixel in the ground area image is filtered using the pixel values ​​of n pixels in the neighborhood to obtain the ground area image after filtering, so as to remove noise points in the ground area image, improve the quality of the ground area image, and avoid the influence of noise points on subsequent contour detection and classification.

[0101] S206: Using a preset gradient algorithm to calculate the gradient of the filtered ground area image to obtain a gradient image.

[0102] In this embodiment, the Sobel operator, Scharr operator, Lapkacian operator, etc. can be used to calculate the gradient of each pixel in the filtered ground area image. For example, the gradient is calculated in the x-direction and the y-direction within the neighborhood of each pixel to obtain a gradient image. For details, reference can be made to the method of calculating the image gradient in the prior art, which will not be described in detail here.

[0103] S207 , performing local peak filtering on the gradient image to obtain a local peak map.

[0104] In an optional embodiment, the local peak image can be obtained by performing local peak filtering on the gradient image in the following manner:

[0105] L = D - a × f (D);

[0106] Among them, L is the local peak image, D is the gradient image, f is the filter function (such as Gaussian, median and other filter functions), and a is the coefficient.

[0107] S208 , binarizing the local peak image based on a pre-configured threshold to obtain a binary image.

[0108] In this embodiment, a pixel threshold for binarization can be pre-configured, and the value of pixels below the pixel threshold is set to 0 (black), and the value of pixels above or equal to the pixel threshold is set to 255 (white). In this way, the local peak image is binarized to obtain a binary image (black and white image), such as Figure 5 The diagram g in the figure is a schematic diagram of a binary image.

[0109] S209: Determine a candidate target based on the binary image, and extract a local peak map of the candidate target from the local peak map.

[0110] In this embodiment, after obtaining a binary image, boundary enhancement processing can be performed on the binary image, and contour detection can be performed on the binary image after boundary enhancement processing to obtain first contours of multiple targets, the number of pixels of the multiple first contours is counted, and the first contours with a pixel number less than a preset number are filtered to obtain a second contour, the minimum circumscribed rectangle of the second contour is determined, and the area of ​​the minimum circumscribed rectangle of the second contour is calculated, the second contour with a minimum circumscribed rectangle area greater than a preset value is determined as a candidate contour, and the target corresponding to the candidate contour in the binary image is determined as a candidate target.

[0111] Exemplarily, the boundary in the binary image may be firstly enhanced by dilation or the like, and then the binary image may be traversed and scanned to find a connected pixel region, and the contour of the connected pixel region may be extracted to obtain a first contour, and the number of pixel points of the first contour may be further counted as the perimeter of the first contour, and the first contours whose perimeters are less than a preset number may be removed to obtain a second contour, and then the minimum bounding rectangle of each second contour may be taken and the area of ​​the minimum bounding rectangle may be calculated (which may be expressed in terms of the number of pixels), and the second contour whose minimum bounding rectangle area is greater than a preset value may be determined as a candidate contour, and the target corresponding to the candidate contour in the binary image may be determined as a candidate target, such as Figure 5 The smallest colored circumscribed rectangle in h is the candidate target, and then the local peak map of each candidate target is extracted from the local peak map obtained in S207.

[0112] S210, inputting the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target.

[0113] In this embodiment, when training the target classification model, a training data set may be first obtained, the training data set including local peak map samples of each target sample and the first category of the target sample. For example, a large number of bottom pit images may be obtained as sample images, and then the local peak map of each target sample is extracted from the sample image through the above-mentioned S202-S209 method to obtain the local peak map sample, and the local peak map sample is manually labeled with the first category (indicating whether it is a foreign body) to obtain the training data set, such as Figure 6 The colored labeled box shown is the target sample, which is labeled with the first category.

[0114] After obtaining the training data set, a target classification model can be constructed and initialized. A local peak map sample of the target sample is randomly extracted and input into the target classification model to obtain a second category of the target sample. The loss value is calculated using the first category and the second category to determine whether the preset training conditions are met. If so, it is determined that the target classification model has completed training. If not, the model parameters of the target classification model are adjusted according to the loss value, and the step of randomly extracting a local peak map sample of the target sample and inputting it into the target classification model is returned.

[0115] Among them, the loss value can be calculated by the mean square error loss function and the cross entropy loss function, the model parameter adjustment can be adjusted by various gradient descent algorithms, and the training condition can be that the loss value is less than the preset value or the number of training times reaches the preset number.

[0116] After the target classification model is trained, it can be deployed in electronic devices, such as embedded electronic devices that control the operation of elevators. After extracting the local peak map of the candidate target, the local peak map of the candidate target is input into the target classification model to obtain the classification result of the candidate target, such as Figure 7 The figure shows the classification result of foreign matter or non-foreign matter after the local peak map of the candidate target is input into the target classification model.

[0117] S211. When the classification result is foreign matter, it is determined that there is foreign matter in the bottom pit of the shaft.

[0118] If the classification result of the candidate target is a foreign object, the candidate target is determined to be a foreign object, and the information of the candidate target can be output. For example, the candidate target detection frame can be output, such as the position and size of the detection frame (e.g. Figure 7 As shown in J in the figure, the position and size of the candidate target identified as a foreign object can be used to determine whether it affects the operation of the elevator and control the operation of the elevator. For example, when a foreign object affects the elevator, the elevator can be controlled to stop running and the elevator can be controlled again after the foreign object is cleared.

[0119] In this embodiment, the original image of the pit area of ​​the shaft is added with prompt points and then input into the SAM model for segmentation to obtain multiple sub-areas, and the comprehensive score of each sub-area is calculated, and the sub-area with the highest comprehensive score is determined as the ground area, and the ground area image is extracted from the original image, and the gradient image of the ground area image is calculated and then further local peak filtering is performed to obtain a local peak map, and the local peak is Figure 2 The binary image is obtained by value processing, and the candidate target is determined after contour detection on the binary image. The local peak map of the candidate target is input into the target classification model to obtain the classification result of the candidate target. When the classification result is a foreign object, it is determined that there is a foreign object in the shaft pit. There is no need to collect images and then reconstruct the three-dimensional scene of the shaft pit and identify the position and height of the object after transforming the world coordinate system. This reduces the calculation amount of foreign object recognition and is suitable for identifying foreign objects in hardware devices with low computing power. It improves the real-time performance of foreign object detection and meets the scene with high real-time requirements for controlling elevators according to foreign object recognition results. In addition, foreign objects are identified by local peak maps through the target classification model without calculating the height. This is suitable for situations where the relationship between foreign objects in the shaft pit is complex, and the accuracy of foreign object recognition is improved.

[0120] Furthermore, by filtering each sub-area through the aspect ratio of its minimum circumscribed rectangle and calculating a comprehensive score through the area, the distance between the center of gravity and the reference point, it is possible to filter out slender sub-areas that are unlikely to be ground areas, and exclude non-ground areas through comprehensive scoring, thereby improving the accuracy of identifying ground areas in the shaft pit.

[0121] Furthermore, filtering the ground area image can remove noise points, and after calculating the gradient to obtain the gradient image, filtering to obtain the local peak map and binarizing the binary image to obtain contour detection to determine the candidate target, which can accurately remove noise points and exclude shadows, noise points, etc. in the original image, thereby improving the accuracy of foreign object recognition.

[0122] Embodiment 3

[0123] Figure 8 This is a schematic diagram of the structure of a foreign body detection device for an elevator shaft pit provided in the third embodiment of the present invention. Figure 8 As shown, the elevator shaft pit foreign body detection device comprises:

[0124] The original image acquisition module 801 is used to acquire the original image of the pit area of ​​the shaft;

[0125] A ground area image extraction module 802 is used to extract a ground area image from the original image;

[0126] A local peak map acquisition module 803 is used to determine a candidate target according to the ground area image and acquire a local peak map of the candidate target;

[0127] A classification module 804 is used to input the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target;

[0128] The foreign matter determination module 805 is used to determine whether there is a foreign matter in the bottom pit of the shaft when the classification result is a foreign matter.

[0129] Optionally, the ground area image extraction module 802 includes:

[0130] A cue point setting unit, used for setting cue points in the original image;

[0131] An image segmentation unit, used for inputting the original image after setting the prompt point into a pre-trained SAM model to obtain multiple sub-regions;

[0132] A ground area determination unit is used to calculate the comprehensive score of each sub-area and determine the sub-area with the largest comprehensive score as the ground area;

[0133] The image capture unit is used to capture the ground area from the original image to obtain a ground area image.

[0134] Optionally, the ground area determination unit includes:

[0135] The aspect ratio calculation subunit is used to determine the minimum bounding rectangle of each sub-region, and calculate the ratio of the long side to the short side of the minimum bounding rectangle to obtain the aspect ratio;

[0136] A candidate sub-region determining sub-unit, used to determine, from among the sub-regions, a sub-region whose aspect ratio is less than a preset threshold value to obtain a candidate sub-region;

[0137] An area and distance calculation subunit, used to calculate the area of ​​each candidate sub-region and the distance from the centroid of each candidate sub-region to a preset reference point;

[0138] The comprehensive score calculation subunit is used to calculate the product of a preset coefficient and the distance, and calculate the difference between the area and the product to obtain a comprehensive score of the candidate sub-region.

[0139] Optionally, the local peak map acquisition module 803 includes:

[0140] A filtering unit, used for filtering the ground area image to obtain a filtered ground area image;

[0141] A gradient image calculation unit, used to calculate the gradient of the filtered ground area image using a preset gradient algorithm to obtain a gradient image;

[0142] A local peak map determining unit, used for performing local peak filtering on the gradient image to obtain a local peak map;

[0143] An image binarization processing unit, used for performing binarization processing on the local peak image based on a pre-configured threshold value;

[0144] The local peak image interception unit is used to determine a candidate target based on the binary image and intercept the local peak image of the candidate target from the local peak image.

[0145] Optionally, the local peak map determining unit is specifically used for:

[0146] The local peak image is obtained by performing local peak filtering on the gradient image in the following manner:

[0147] L = D - a × f (D);

[0148] Among them, L is the local peak image, D is the gradient image, f is the filter function, and a is the coefficient.

[0149] Optionally, the local peak image interception unit includes:

[0150] A contour detection subunit is used to perform boundary enhancement processing on the binary image, and perform contour detection on the binary image after boundary enhancement processing to obtain first contours of multiple targets;

[0151] A contour screening subunit, used for counting the number of pixels of a plurality of first contours, and filtering the first contours whose number of pixels is less than a preset number, to obtain a second contour;

[0152] A contour area calculation subunit, used to determine the minimum circumscribed rectangle of the second contour and calculate the area of ​​the minimum circumscribed rectangle of the second contour;

[0153] The candidate target determination subunit is used to determine a second contour whose minimum circumscribed rectangular area is greater than a preset value as a candidate contour, and determine a target corresponding to the candidate contour in the binary image as a candidate target.

[0154] Optionally, the target classification model is trained by the following modules:

[0155] A training data acquisition module, used to acquire a training data set, wherein the training data set includes a local peak map sample of each target sample and a first category of the target sample;

[0156] A model building and initialization module, used to build a target classification model and initialize the target classification model;

[0157] A training data input module, used for randomly extracting local peak map samples of target samples and inputting them into the target classification model to obtain a second category of the target samples;

[0158] A loss value calculation module, used for calculating a loss value using the first category and the second category;

[0159] The training condition judgment module is used to judge whether the preset training conditions are met; if so, the training completion determination module is executed; if not, the model parameter adjustment module is executed;

[0160] A training completion determination module, used to determine that the target classification model has completed training;

[0161] A model parameter adjustment module is used to adjust the model parameters of the target classification model according to the loss value and return it to the training data input module.

[0162] The elevator shaft pit foreign object detection device provided in the embodiment of the present invention can execute the elevator shaft pit foreign object detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0163] Embodiment 4

[0164] Fig. 9A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0165] like Fig. 9 As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0166] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0167] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the elevator shaft pit foreign body detection method.

[0168] In some embodiments, the elevator shaft pit foreign body detection method can be implemented as a computer program, which is tangibly contained in a computer readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the elevator shaft pit foreign body detection method described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to perform the elevator shaft pit foreign body detection method in any other appropriate manner (e.g., by means of firmware).

[0169] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0171] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0173] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0174] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0175] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0176] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting foreign matter in an elevator shaft pit, characterized in that: include: Obtaining the original image of the shaft pit area; Extracting a ground area image from the original image; Determine a candidate target according to the ground area image, and obtain a local peak map of the candidate target; Inputting the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target; When the classification result is foreign matter, it is determined that there is foreign matter in the shaft bottom pit.

2. The method according to claim 1, characterized in that Extracting a ground area image from the original image includes: Setting a cue point in the original image; The original image after setting the cue point is input into the pre-trained SAM model to obtain multiple sub-regions; Calculate the comprehensive score of each sub-area, and determine the sub-area with the largest comprehensive score as the ground area; The ground area is intercepted from the original image to obtain a ground area image.

3. The method according to claim 2, characterized in that A composite score is calculated for each sub-area, including: Determine the minimum bounding rectangle of each sub-region, and calculate the ratio of the long side to the short side of the minimum bounding rectangle to obtain the aspect ratio; Determine, from among the sub-regions, a sub-region whose aspect ratio is less than a preset threshold to obtain a candidate sub-region; Calculating the area of ​​each candidate sub-region and the distance from the centroid of each candidate sub-region to a preset reference point; The product of a preset coefficient and the distance is calculated, and the difference between the area and the product is calculated to obtain a comprehensive score of the candidate sub-region.

4. The method according to claim 1, characterized in that Determining a candidate target according to the ground area image and obtaining a local peak map of the candidate target includes: Performing filtering processing on the ground area image to obtain a filtered ground area image; A preset gradient algorithm is used to calculate the gradient of the filtered ground area image to obtain a gradient image; Performing local peak filtering on the gradient image to obtain a local peak map; A binary image of a local peak image subjected to binarization processing based on a pre-configured threshold; A candidate target is determined based on the binary image, and a local peak map of the candidate target is intercepted from the local peak map.

5. The method according to claim 4, characterized in that Performing local peak filtering on the gradient image to obtain a local peak map includes: The local peak image is obtained by performing local peak filtering on the gradient image in the following manner: L = D - a × f (D); Among them, L is the local peak image, D is the gradient image, f is the filter function, and a is the coefficient.

6. The method according to claim 4, characterized in that Determining a candidate target based on the binary image includes: Performing boundary enhancement processing on the binary image, and performing contour detection on the binary image after the boundary enhancement processing to obtain first contours of multiple targets; Counting the number of pixels of the plurality of first contours, and filtering the first contours whose number of pixels is less than a preset number, to obtain a second contour; Determine a minimum circumscribed rectangle of the second contour, and calculate the area of ​​the minimum circumscribed rectangle of the second contour; A second contour whose minimum circumscribed rectangle area is greater than a preset value is determined as a candidate contour, and an object corresponding to the candidate contour in the binary image is determined as a candidate object.

7. The method according to any one of claims 1 to 6, characterized in that: The target classification model is trained by the following steps: Acquire a training data set, wherein the training data set includes local peak map samples of each target sample and a first category of the target sample; Constructing a target classification model and initializing the target classification model; Randomly extracting a local peak map sample of the target sample and inputting it into the target classification model to obtain a second category of the target sample; calculating a loss value using the first category and the second category; Determine whether the preset training conditions are met; If so, determining that the target classification model has completed training; If not, the model parameters of the target classification model are adjusted according to the loss value, and the step of randomly extracting local peak map samples of the target samples and inputting them into the target classification model is returned.

8. An elevator shaft pit foreign body detection device, characterized in that: include: The original image acquisition module is used to acquire the original image of the pit area of ​​the shaft; A ground area image extraction module, used to extract a ground area image from the original image; A local peak image acquisition module, used to determine a candidate target according to the ground area image and acquire a local peak image of the candidate target; A classification module, used for inputting the local peak map of the candidate target into a pre-trained target classification model to obtain a classification result of the candidate target; A foreign matter determination module is used to determine that there is a foreign matter in the bottom pit of the shaft when the classification result is a foreign matter.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the elevator shaft pit foreign object detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the elevator shaft pit foreign object detection method according to any one of claims 1 to 7 when executed.

Images