Recognition method and system for opening and closing state of bin gate, electronic equipment and storage medium

Abstract

The invention provides a bin gate opening and closing state identification method and system, electronic equipment and a storage medium, and relates to the technical field of monitoring image processing. Comprising the following steps: acquiring target image information acquired by monitoring equipment; setting at least one algorithm recognition area according to the target image information; performing image recognition, depth estimation and coordinate transformation on the algorithm recognition area to obtain bin gate contour point three-dimensional point cloud coordinates and wall surface contour point three-dimensional point cloud coordinates corresponding to the bin gate contour points and the wall surface contour points respectively; determining a warehouse door normal vector of a plane where the warehouse door is located and a wall surface normal vector of a plane where the wall surface is located according to the warehouse door contour point three-dimensional point cloud coordinates and the wall surface contour point three-dimensional point cloud coordinates; when it is judged that the plane where the bin door is located is parallel to the plane where the wall face is located according to the bin door normal vector and the wall face normal vector, it is determined that the bin door is in a completely closed state; and under the condition that the included angle between the normal vector of the bin gate and the normal vector of the wall surface is judged to be greater than a preset threshold value, determining that the bin gate is in an open state.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring image processing, and in particular to a method for identifying the switch status of a warehouse door, a system thereof, an electronic device and a storage medium. Background Art

[0002] The warehouse door is a door used in warehouses or cabins, mainly used for the entry and exit of goods. In order to ensure the safety of goods in the warehouse, it is also necessary to monitor the opening and closing status of the warehouse door in real time to prevent illegal personnel from entering and leaving the warehouse abnormally. Identification of the opening and closing status of the warehouse door is an important measure to supervise warehouse safety and can effectively avoid the risk of cargo theft.

[0003] At present, the door status is generally identified based on the door key points or door classification. The identification method based on the door key points is to determine whether the door is open or closed based on the angle between at least two key points on the upper edge of the door and the horizontal line. However, the inventors found that the identification method based on the door key points has high requirements for the imaging image, and the number of key points of single doors and double doors is not consistent, resulting in poor versatility. The identification method based on door classification requires the collection of training data of different door types, which has problems such as large workload and inability to calculate the door opening angle and poor flexibility. Summary of the invention

[0004] The present invention provides a method and system for identifying the switch status of a warehouse door, an electronic device and a storage medium, which are used to solve the problems of the current method for identifying the switch status of a warehouse door, such as high cost, poor versatility, large workload, inability to calculate the opening angle of the warehouse door and poor flexibility of use.

[0005] According to one aspect of the present invention, the present invention provides a method for identifying the switch state of a warehouse door. The identification method includes: obtaining target image information collected by a monitoring device; setting at least one algorithm identification area according to the target image information; performing image recognition, depth estimation and coordinate conversion on the algorithm identification area to obtain the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the warehouse door contour points and the wall contour points respectively; determining the warehouse door normal vector of the plane where the warehouse door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points; determining that the warehouse door is in a completely closed state when it is determined that the plane where the warehouse door is located is parallel to the plane where the wall is located according to the warehouse door normal vector and the wall normal vector; determining that the warehouse door is in an open state when it is determined that the angle between the warehouse door normal vector and the wall normal vector is greater than a preset threshold.

[0006] According to some embodiments of the present invention, obtaining target image information collected by a monitoring device includes: obtaining initial image information collected by the monitoring device; calibrating parameters of the initial image information; and performing distortion correction on the initial image information according to parameters obtained by the parameter calibration to obtain target image information.

[0007] According to some embodiments of the present invention, image recognition, depth estimation and coordinate conversion are performed on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the door contour points and the wall contour points respectively, including: performing image segmentation and recognition on the algorithm recognition area to obtain the door contour points and the wall contour points; obtaining the door contour point depth values ​​and the wall contour point depth values ​​corresponding to the door contour points and the wall contour points based on the depth estimation; and calculating the door contour point three-dimensional point cloud coordinates and the wall contour point three-dimensional point cloud coordinates according to the door contour point depth values ​​and the wall contour point depth values.

[0008] According to some embodiments of the present invention, determining the door normal vector of the plane where the door is located and the wall normal vector of the plane where the wall is located based on the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points includes: constructing the door plane equation based on the three-dimensional point cloud coordinates of the door contour points; constructing the wall plane equation based on the three-dimensional point cloud coordinates of the wall contour points; and respectively determining the door normal vector and the wall normal vector based on the door plane equation and the wall plane equation.

[0009] According to another aspect of the present invention, the present invention provides a system for identifying the switch state of a warehouse door. The identification system includes an image processing module, a coordinate processing module and an information processing module. The image processing module obtains the target image information collected by the monitoring device, and sets at least one algorithm identification area according to the target image information; the coordinate processing module performs image recognition, depth estimation and coordinate conversion on the algorithm identification area to obtain the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the warehouse door contour points and the wall contour points respectively, and determines the warehouse door normal vector of the plane where the warehouse door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points; the information processing module determines that the warehouse door is in a completely closed state when it is determined that the plane where the warehouse door is located is parallel to the plane where the wall is located according to the warehouse door normal vector and the wall normal vector, and determines that the warehouse door is in an open state when it is determined that the angle between the warehouse door normal vector and the wall normal vector is greater than a preset threshold.

[0010] According to some embodiments of the present invention, the image processing module obtains initial image information collected by the monitoring device; the image processing module performs parameter calibration on the initial image information; the image processing module performs distortion correction on the initial image information according to the parameters obtained by the parameter calibration to obtain target image information.

[0011] According to some embodiments of the present invention, the coordinate processing module performs image segmentation and recognition on the algorithm identification area to obtain door contour points and wall contour points; the coordinate processing module obtains the door contour point depth values ​​and wall contour point depth values ​​corresponding to the door contour points and wall contour points based on depth estimation; the coordinate processing module calculates the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points according to the door contour point depth values ​​and the wall contour point depth values.

[0012] According to some embodiments of the present invention, the coordinate processing module constructs the plane equation of the warehouse door based on the three-dimensional point cloud coordinates of the warehouse door contour points; the coordinate processing module constructs the plane equation of the wall based on the three-dimensional point cloud coordinates of the wall contour points; the coordinate processing module determines the warehouse door normal vector and the wall normal vector based on the warehouse door plane equation and the wall plane equation respectively.

[0013] According to another aspect of the present invention, an electronic device is provided, which includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors can implement the identification method described above.

[0014] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the identification method described above can be implemented.

[0015] The present invention performs image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points respectively corresponding to the warehouse door contour points and the wall contour points, and further obtains the warehouse door normal vector and the wall normal vector according to the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points, and determines the state of the warehouse door according to the relationship between the warehouse door normal vector and the wall normal vector.

[0016] The present invention replaces the technical solution in the prior art of judging the state of the warehouse door based on the angle between the key point on the upper edge of the warehouse door and the horizontal line with the technical solution in the present invention of judging the state of the warehouse door based on the angle between the normal vector of the warehouse door and the normal vector of the wall. The present invention is applicable to both single-door and double-door application scenarios.

[0017] The recognition method provided by the present invention can accurately recognize the state of the warehouse door based on image processing and coordinate conversion in a single monitoring device scenario. The present invention has the characteristics of low cost, strong versatility and high recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flow chart showing an identification method according to an embodiment of the present invention;

[0020] Figure 2 Another flow chart showing the identification method according to the embodiment of the present invention;

[0021] Figure 3 Another flow chart showing the identification method according to the embodiment of the present invention;

[0022] Figure 4 Another flow chart showing the identification method according to the embodiment of the present invention;

[0023] Figure 5 A schematic diagram showing an identification system according to an embodiment of the present invention.

[0024] Description of reference numerals:

[0025] Recognition system 1; image processing module 10; coordinate processing module 20; information processing module 30. DETAILED DESCRIPTION

[0026] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0027] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other modes, components, materials, devices, etc. may be adopted. In these cases, known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0028] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0029] The terms "first", "second" and the like in the specification, claims and the above drawings of the present invention are used to distinguish different objects rather than to describe a specific order.

[0030] The technical solution of the present invention is described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] The full names of the English abbreviations involved in the present invention and their Chinese meanings are:

[0032] RTSP: Real Time Streaming Protocol, real-time streaming protocol;

[0033] SSD: Single Shot MultiBox Detector, a target detection algorithm based on deep learning;

[0034] YOLO: You Only Look Once, a real-time target detection algorithm based on deep learning;

[0035] RetinaNet: a single-stage object detection model.

[0036] RT-DETR: Real-Time Detection Transformer, real-time end-to-end object detection model;

[0037] RANSAC: RANdom SAmple Consensus, random sampling consensus algorithm.

[0038] According to one aspect of the present invention, the present invention provides a method for identifying the switch status of a warehouse door, which is used to identify whether the warehouse door is in an open state or a closed state, so as to control the status of the warehouse door in real time.

[0039] According to an example embodiment, Figure 1 As shown, the recognition method may include steps S100-S600. Exemplarily, the recognition method may be executed by a door switch state recognition system having computing capability.

[0040] In step S100, the recognition system obtains target image information collected by the monitoring device.

[0041] For example, the monitoring device can be a monocular camera, which is installed at a preset monitoring position according to monitoring requirements (e.g., the deflection angle between the monitoring device and the normal direction of the warehouse door is no more than 45°), and is used to collect monitoring images at the warehouse door. The monitoring device has a fixed installation position, monitoring angle, focal length, etc.

[0042] Alternatively, if Figure 2 As shown, step S100 may also include steps S110-S130.

[0043] In step S110 , the recognition system obtains initial image information collected by the monitoring device.

[0044] For example, the recognition system is connected to a monocular camera for communication, obtains an RTSP video stream from the monocular camera, and decodes and processes the RTSP video stream to obtain initial image information at the warehouse door.

[0045] Exemplarily, the initial image information may be color image information or grayscale image information.

[0046] In step S120, the recognition system performs parameter calibration on the initial image information.

[0047] In step S130, the recognition system performs distortion correction on the initial image information according to the parameters obtained by parameter calibration to obtain target image information.

[0048] For example, parameter calibration includes internal parameter calibration. For example, the recognition system performs internal parameter calibration based on a pre-selected calibration method (such as Zhang's calibration method), and then performs distortion correction on the color image information according to the obtained internal parameter matrix and distortion coefficient, thereby obtaining the distortion-corrected image information (i.e., the target image information).

[0049] In step S200, the recognition system sets at least one algorithm recognition area according to the target image information.

[0050] For example, the recognition system responds to user operation instructions and sets one or more algorithm recognition areas based on target image information according to user needs. The algorithm recognition area can be a user-defined area with monitoring and recognition significance, that is, an alarm will be issued only when a person is recognized in the algorithm recognition area.

[0051] Exemplarily, the recognition system may generate a polygonal area based on coordinates of points within the algorithm recognition area.

[0052] In step S300, the recognition system performs image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the door contour points and the wall contour points respectively.

[0053] For example, the recognition system can segment and identify the door contour points and wall contour points by performing image segmentation and recognition processing on the algorithm recognition area, and then obtain the door contour point coordinates and wall contour point coordinates corresponding to the door contour points and wall contour points.

[0054] The recognition system can obtain the corresponding three-dimensional point cloud coordinates of the door contour points and the wall contour points through depth estimation and coordinate conversion processing.

[0055] Exemplarily, the three-dimensional point cloud coordinates are (x, y, z).

[0056] Alternatively, if Figure 3 As shown, step S300 may also include steps S310-S330.

[0057] In step S310, the recognition system performs image segmentation and recognition on the algorithm recognition area to obtain the door contour points and wall contour points.

[0058] For example, the recognition system determines the door contour points and wall contour points in the target image information based on the image segmentation recognition technology, and then obtains the corresponding door contour point coordinates and wall contour point coordinates.

[0059] Exemplarily, the image segmentation and recognition technology includes but is not limited to SSD, YOLO, RetinaNet or RT-DETR, etc., and the present invention is not limited to this.

[0060] In step S320, the recognition system obtains the depth values ​​of the door contour points and the wall contour points corresponding to the door contour points and the wall contour points based on depth estimation.

[0061] For example, the recognition system performs monocular depth estimation based on the target image information, and can obtain the depth values ​​of all pixels in the target image information. Then, the recognition system determines the corresponding depth values ​​of the door contour points and the wall contour points according to the coordinates of the door contour points and the wall contour points.

[0062] In step S330, the recognition system calculates the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points according to the depth values ​​of the door contour points and the wall contour points.

[0063] For example, the recognition system converts the depth values ​​of the door contour points and the wall contour points into three-dimensional point cloud coordinates of the door contour points and the wall contour points.

[0064] Exemplarily, the calculation formula of the three-dimensional point cloud coordinates (x, y, z) can be:

[0065]

[0066] Among them, (u, ν) is the coordinate of the pixel point, z c is the depth value of the pixel (u, ν),

[0067] (u0, ν0) are the coordinates of the principal point; f x The length of the focal length in the x-axis direction is described in pixels; f y The length of the focal length in the y-axis direction is described in pixels; depth_scale is the scale factor corresponding to the depth map (the ratio of the value stored in the depth map to the actual depth).

[0068] In step S400, the recognition system determines the door normal vector of the plane where the door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points.

[0069] For example, the recognition system determines the door normal vector of the plane where the door is located based on the three-dimensional point cloud coordinates of the door contour points. And determine the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the wall contour points

[0070] Alternatively, if Figure 4 As shown, step S400 may also include steps S410-S430.

[0071] In step S410, the recognition system constructs the plane equation of the warehouse door based on the three-dimensional point cloud coordinates of the warehouse door contour points.

[0072] For example, the plane equation of the door can be:

[0073] A1x+B1y+C1z+D1=0;

[0074] Among them, A1, B1, C1 and D1 are the coefficients of the warehouse door plane equation.

[0075] Exemplarily, the fitting plane may be solved based on the RANSAC method, so that the coefficients A1, B1, C1 and D1 may be obtained.

[0076] In step S420, the recognition system constructs a wall plane equation based on the three-dimensional point cloud coordinates of the wall contour points.

[0077] For example, the wall plane equation can be:

[0078] A2x+B2y+C2z+D2=0;

[0079] Among them, A2, B2, C2 and D2 are the coefficients of the wall plane equation.

[0080] Exemplarily, the fitting plane may be solved based on the RANSAC method, so that the coefficients A2, B2, C2 and D2 may be obtained.

[0081] In step S430, the recognition system determines the door normal vector based on the door plane equation and the wall plane equation. and the wall normal vector

[0082] For example, the door normal vector Wall Normal

[0083] In step S500, the recognition system uses the door normal vector and the wall normal vector When it is determined that the plane where the door is located is parallel to the plane where the wall is located, it is determined that the door is in a completely closed state.

[0084] For example, the recognition system uses the door normal vector and the wall normal vector Determine whether the plane where the door is located is parallel to the plane where the wall is located.

[0085] Exemplarily, when A1 / A2=B1 / B2=C1 / C2, the recognition system determines that the plane where the door is located is parallel to the plane where the wall is located.

[0086] In step S600, when the recognition system determines that the angle between the door normal vector and the wall normal vector is greater than a preset threshold, the recognition system determines that the door is in an open state.

[0087] For example, the door normal vector and the wall normal vector The angle between them is θ, such as:

[0088]

[0089] When θ is greater than a preset threshold, the recognition system determines that the door is open and issues an alarm. In this way, the present invention can map the angle between the actual door and the actual wall through the angle between the door normal vector and the wall normal vector, thereby accurately determining the open and closed state of the door.

[0090] Through the above embodiments, the present invention performs image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points respectively corresponding to the warehouse door contour points and the wall contour points, and further obtains the warehouse door normal vector and the wall normal vector according to the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points, and determines the state of the warehouse door according to the relationship between the warehouse door normal vector and the wall normal vector.

[0091] The present invention replaces the technical solution in the prior art of judging the state of the warehouse door based on the angle between the key point on the upper edge of the warehouse door and the horizontal line with the technical solution in the present invention of judging the state of the warehouse door based on the angle between the normal vector of the warehouse door and the normal vector of the wall. The present invention is applicable to both single-door and double-door application scenarios.

[0092] The recognition method provided by the present invention can accurately recognize the state of the warehouse door based on image processing and coordinate conversion in a single monitoring device scenario. The present invention has the characteristics of low cost, strong versatility and high recognition accuracy.

[0093] According to another aspect of the present invention, the present invention provides a system for identifying the switch status of a warehouse door, which is used to identify whether the warehouse door is in an open state or a closed state, so as to control the status of the warehouse door in real time.

[0094] According to an example embodiment, Figure 5 As shown, the recognition system 1 includes an image processing module 10 , a coordinate processing module 20 and an information processing module 30 .

[0095] The image processing module 10 obtains target image information collected by the monitoring device.

[0096] For example, the monitoring device can be a monocular camera, which is installed at a preset monitoring position according to monitoring requirements (e.g., the deflection angle between the monitoring device and the normal direction of the warehouse door is no more than 45°), and is used to collect monitoring images at the warehouse door. The monitoring device has a fixed installation position, monitoring angle, focal length, etc.

[0097] Optionally, the image processing module 10 obtains initial image information collected by the monitoring device.

[0098] For example, the image processing module 10 is connected to a monocular camera for communication, obtains an RTSP video stream from the monocular camera, and decodes and processes the RTSP video stream to obtain initial image information at the warehouse door.

[0099] Exemplarily, the initial image information may be color image information or grayscale image information.

[0100] The image processing module 10 performs parameter calibration on the initial image information.

[0101] The image processing module 10 performs distortion correction on the initial image information according to the parameters obtained by parameter calibration to obtain target image information.

[0102] For example, parameter calibration includes internal parameter calibration. For example, the image processing module 10 performs internal parameter calibration based on a pre-selected calibration method (such as Zhang's calibration method), and then performs distortion correction on the color image information according to the obtained internal parameter matrix and distortion coefficient, thereby obtaining the image information after distortion correction (i.e., the target image information).

[0103] According to an example embodiment, the image processing module 10 sets at least one algorithm recognition area according to target image information.

[0104] For example, the image processing module 10 responds to the user's operation instruction and sets one or more algorithm recognition areas based on the target image information according to the user's needs. The algorithm recognition area can be a user-defined area with monitoring and recognition significance, that is, an alarm prompt will be issued only when a person is recognized in the algorithm recognition area.

[0105] Exemplarily, the image processing module 10 may generate a polygonal area based on the coordinates of points within the algorithm recognition area.

[0106] According to an exemplary embodiment, the coordinate processing module 20 performs image recognition, depth estimation and coordinate conversion on the algorithm identification area to obtain the three-dimensional point cloud coordinates of the door contour points and the wall contour points corresponding to the door contour points and the wall contour points respectively.

[0107] For example, the coordinate processing module 20 can segment and identify the door contour points and wall contour points by performing image segmentation and recognition processing on the algorithm recognition area, and then obtain the door contour point coordinates and wall contour point coordinates corresponding to the door contour points and wall contour points.

[0108] The coordinate processing module 20 processes the door contour point coordinates and the wall contour point coordinates through depth estimation and coordinate conversion to obtain the corresponding door contour point three-dimensional point cloud coordinates and wall contour point three-dimensional point cloud coordinates.

[0109] Exemplarily, the three-dimensional point cloud coordinates are (x, y, z).

[0110] Optionally, the coordinate processing module 20 performs image segmentation and recognition on the algorithm recognition area to obtain the door contour points and the wall contour points.

[0111] For example, the coordinate processing module 20 determines the door contour points and the wall contour points in the target image information based on the image segmentation and recognition technology, and then obtains the corresponding door contour point coordinates and wall contour point coordinates.

[0112] Exemplarily, the image segmentation and recognition technology includes but is not limited to SSD, YOLO, RetinaNet or RT-DETR, etc., and the present invention is not limited to this.

[0113] The coordinate processing module 20 obtains the depth values ​​of the door contour points and the wall contour points corresponding to the door contour point coordinates and the wall contour point coordinates based on the depth estimation.

[0114] For example, the coordinate processing module 20 performs monocular depth estimation based on the target image information to obtain the depth values ​​of all pixels in the target image information. Then, the coordinate processing module 20 determines the corresponding depth values ​​of the door contour points and the wall contour points according to the coordinates of the door contour points and the wall contour points.

[0115] The coordinate processing module 20 calculates the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points according to the depth values ​​of the door contour points and the wall contour points.

[0116] For example, the coordinate processing module 20 converts the depth values ​​of the door contour points and the wall contour points into three-dimensional point cloud coordinates of the door contour points and three-dimensional point cloud coordinates of the wall contour points.

[0117] Exemplarily, the calculation formula of the three-dimensional point cloud coordinates (x, y, z) can be:

[0118]

[0119] Among them, (u, ν) is the coordinate of the pixel point, z c is the depth value of the pixel (u, ν),

[0120] (u0, ν0) are the coordinates of the principal point; f x The length of the focal length in the x-axis direction is described in pixels; f y The length of the focal length in the y-axis direction is described in pixels; depth_scale is the scale factor corresponding to the depth map (the ratio of the value stored in the depth map to the actual depth).

[0121] According to an exemplary embodiment, the coordinate processing module 20 determines the door normal vector of the plane where the door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points.

[0122] For example, the coordinate processing module 20 determines the door normal vector of the plane where the door is located according to the three-dimensional point cloud coordinates of the door contour points. And determine the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the wall contour points

[0123] Optionally, the coordinate processing module 20 constructs the plane equation of the warehouse door based on the three-dimensional point cloud coordinates of the warehouse door contour points.

[0124] For example, the plane equation of the door can be:

[0125] A1x+B1y+C1z+D1=0;

[0126] Among them, A1, B1, C1 and D1 are the coefficients of the warehouse door plane equation.

[0127] Exemplarily, the fitting plane may be solved based on the RANSAC method, so that the coefficients A1, B1, C1 and D1 may be obtained.

[0128] The coordinate processing module 20 constructs the wall plane equation based on the three-dimensional point cloud coordinates of the wall contour points.

[0129] For example, the wall plane equation can be:

[0130] A2x+B2y+C2z+D2=0;

[0131] Among them, A2, B2, C2 and D2 are the coefficients of the wall plane equation.

[0132] Exemplarily, the fitting plane may be solved based on the RANSAC method, so that the coefficients A2, B2, C2 and D2 may be obtained.

[0133] The coordinate processing module 20 determines the door normal vector based on the door plane equation and the wall plane equation. and the wall normal vector

[0134] For example, the door normal vector Wall Normal

[0135] According to an exemplary embodiment, the information processing module 30 calculates the door normal vector according to the door normal vector. and the wall normal vector When it is determined that the plane where the door is located is parallel to the plane where the wall is located, it is determined that the door is in a completely closed state.

[0136] For example, the information processing module 30 calculates the door normal vector and the wall normal vector Determine whether the plane where the door is located is parallel to the plane where the wall is located.

[0137] Exemplarily, when A1 / A2=B1 / B2=C1 / C2, the information processing module 30 determines that the plane where the door is located is parallel to the plane where the wall is located.

[0138] According to an exemplary embodiment, the information processing module 30 determines that the door is in an open state when it is determined that the angle between the door normal vector and the wall normal vector is greater than a preset threshold.

[0139] For example, the door normal vector and the wall normal vector The angle between them is θ, such as:

[0140]

[0141] When θ is greater than a preset threshold, the information processing module 30 determines that the door is in an open state and issues an alarm. In this way, the present invention can map the angle between the actual door and the actual wall through the angle between the door normal vector and the wall normal vector, thereby accurately determining the open and closed state of the door.

[0142] Through the above embodiments, the present invention performs image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points respectively corresponding to the warehouse door contour points and the wall contour points, and further obtains the warehouse door normal vector and the wall normal vector according to the three-dimensional point cloud coordinates of the warehouse door contour points and the three-dimensional point cloud coordinates of the wall contour points, and determines the state of the warehouse door according to the relationship between the warehouse door normal vector and the wall normal vector.

[0143] The present invention replaces the technical solution in the prior art of judging the state of the warehouse door based on the angle between the key point on the upper edge of the warehouse door and the horizontal line with the technical solution in the present invention of judging the state of the warehouse door based on the angle between the normal vector of the warehouse door and the normal vector of the wall. The present invention is applicable to both single-door and double-door application scenarios.

[0144] The recognition method provided by the present invention can accurately recognize the state of the warehouse door based on image processing and coordinate conversion in a single monitoring device scenario. The present invention has the characteristics of low cost, strong versatility and high recognition accuracy.

[0145] According to another aspect of the present invention, an electronic device is provided, which includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors can implement the identification method described above.

[0146] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the identification method described above can be implemented.

[0147] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions of the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying the switch status of a door, characterized in that: include: Obtain target image information collected by monitoring equipment; Setting at least one algorithm recognition area according to the target image information; Performing image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the door contour points and the wall contour points respectively; Determine the door normal vector of the plane where the door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points; When it is determined that the plane where the door is located is parallel to the plane where the wall is located according to the door normal vector and the wall normal vector, determining that the door is in a fully closed state; When it is determined that the angle between the door normal vector and the wall normal vector is greater than a preset threshold, it is determined that the door is in an open state.

2. The identification method according to claim 1, characterized in that: The obtaining of target image information collected by the monitoring device includes: Acquiring initial image information collected by the monitoring device; Performing parameter calibration on the initial image information; The initial image information is subjected to distortion correction according to the parameters obtained by parameter calibration to obtain the target image information.

3. The identification method according to claim 1, characterized in that: Performing image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the door contour points and the wall contour points respectively includes: Performing image segmentation and recognition on the algorithm recognition area to obtain door contour points and wall contour points; Acquire the door contour point depth value and the wall contour point depth value corresponding to the door contour point and the wall contour point based on the depth estimation; The three-dimensional point cloud coordinates of the door outline point and the three-dimensional point cloud coordinates of the wall outline point are calculated according to the depth value of the door outline point and the depth value of the wall outline point.

4. The identification method according to claim 1, characterized in that: The method of determining the door normal vector of the plane where the door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points comprises: Constructing a door plane equation based on the three-dimensional point cloud coordinates of the door contour points; Constructing a wall plane equation based on the three-dimensional point cloud coordinates of the wall contour points; The door normal vector and the wall normal vector are determined based on the door plane equation and the wall plane equation respectively.

5. A door switch status recognition system, characterized in that: include: An image processing module, which obtains target image information collected by the monitoring device and sets at least one algorithm recognition area according to the target image information; A coordinate processing module performs image recognition, depth estimation and coordinate conversion on the algorithm recognition area to obtain the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points corresponding to the door contour points and the wall contour points respectively, and determines the door normal vector of the plane where the door is located and the wall normal vector of the plane where the wall is located according to the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points; The information processing module determines that the warehouse door is in a completely closed state when it is determined that the plane where the warehouse door is located is parallel to the plane where the wall is located based on the warehouse door normal vector and the wall normal vector; and determines that the warehouse door is in an open state when it is determined that the angle between the warehouse door normal vector and the wall normal vector is greater than a preset threshold.

6. The identification system according to claim 5, characterized in that: The image processing module obtains initial image information collected by the monitoring device; The image processing module performs parameter calibration on the initial image information; The image processing module performs distortion correction on the initial image information according to the parameters obtained by parameter calibration to obtain the target image information.

7. The identification system according to claim 5, characterized in that: The coordinate processing module performs image segmentation and recognition on the algorithm recognition area to obtain the door contour points and wall contour points; The coordinate processing module obtains the depth values ​​of the door contour points and the wall contour points corresponding to the door contour points and the wall contour points based on depth estimation; The coordinate processing module calculates the three-dimensional point cloud coordinates of the door contour points and the three-dimensional point cloud coordinates of the wall contour points according to the depth values ​​of the door contour points and the depth values ​​of the wall contour points.

8. The identification system according to claim 5, characterized in that: The coordinate processing module constructs the door plane equation based on the three-dimensional point cloud coordinates of the door contour points; The coordinate processing module constructs a wall plane equation based on the three-dimensional point cloud coordinates of the wall contour points; The coordinate processing module determines the door normal vector and the wall normal vector based on the door plane equation and the wall plane equation respectively.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the identification method as described in any one of claims 1-4.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the identification method according to any one of claims 1 to 4 is implemented.

Images