Automatic door control system based on machine vision

Abstract

The present invention discloses an automatic door control system based on machine vision, including a video access module, a video processing module and an execution control module, wherein the video access module is used to extract video information from a camera or a system with a camera; the video processing module is used to demarcate a detection area in the video information, and identify objects in the detection area, and determine the position of a human body when the human body is identified; the execution control module is used to demarcate an execution area in the middle of the detection area, and when the position of the human body identified by the video processing module is located in the execution area, a control instruction to open the automatic door is issued. In the present invention, by extracting video information near the automatic door from the existing system with a camera, there is no need to set up a camera separately, which can save system costs; by dividing the detection area and the execution area, the control of the automatic door is more accurate and reasonable.

Description

Technical Field

[0001] The invention belongs to the field of automatic door control, and in particular relates to an automatic door control system based on machine vision. Background Art

[0002] At present, automatic doors are widely used in public places such as shopping malls and office buildings. However, in recent years, safety accidents such as "injury" by automatic doors have occurred frequently. In the mildest cases, people are knocked down or pinched, and in the more serious cases, their lives are even endangered. The main reason is that the existing automatic door sensor device obtains less information and can only detect before people enter the automatic door. Moreover, it is designed for adults, so it cannot detect when people pass through the automatic door and after they pass through the automatic door. This type of sensor has a single detection content. With the development of intelligent building technology, it has failed to meet people's higher requirements for the stability, safety and intelligence of automatic doors. In the CN102747919B patent, although a machine vision-based door automatic control system is recorded, the camera used to obtain images must be installed on the top of the automatic door, that is, a dedicated camera needs to be configured for the automatic door control system, which greatly increases the cost compared with the traditional automatic door control system and is difficult to apply on a large scale. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide an automatic door control system based on machine vision.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] An automatic door control system based on machine vision, comprising:

[0006] A video access module, used for extracting video information from a camera or a system having a camera;

[0007] A video processing module, used to define a detection area in the video information, identify objects in the detection area, and determine the position of a human body when the human body is identified; and

[0008] The execution control module is used to define an execution area in the middle of the detection area, and when the position of the human body identified by the video processing module is located in the execution area, issue a control instruction to open the automatic door.

[0009] Furthermore, the video acquisition module is a video splitter, and the video splitter is used to separate a path of video information from a system with a camera and transmit it to the video processing module.

[0010] Furthermore, the video processing module defines a rectangular area near the automatic door in the video information as a detection area, defines a rectangular area where the automatic door is located as an execution area, and establishes a plane coordinate system in the video information to calibrate the coordinate range of the detection area and the coordinate range of the execution area respectively;

[0011] After identifying the human body, the video processing module first identifies the bounding box area of ​​the human body, then accurately identifies the feet of the human body in the bounding box area, and obtains the coordinates of the feet. When the coordinates of at least one foot of the human body are within the coordinate range of the execution area in a predetermined number of consecutive frames, it is determined that the position of the human body is located in the execution area.

[0012] Furthermore, the execution control module is provided with a mark flag_a for indicating whether the position of a human body is located in the execution area. When it is detected that the position of a human body is located in the execution area, flag_a=1; otherwise, flag_a=0; when flag_a=1, the execution control module issues a control instruction to open the automatic door, until flag_a=0, the execution control module issues a control instruction to close the automatic door; the execution control module is also provided with a mark flag_b for indicating whether a person stays in the execution area for a long time. When it is detected that the position of a human body is located in the execution area, the time the person stays in the execution area is also counted. When the timing time reaches a predetermined time threshold, the execution control module sets flag_b=1; if the position of the person leaves the execution area before the timing time reaches the predetermined time threshold, the timing of the person is stopped and the timing time of the person is restored to the initial value.

[0013] Furthermore, after identifying the human body, the video processing module also determines whether the human body is a child. When the human body is a child, it further detects whether there is an adult within a preset distance threshold range. If there is no adult, the child is determined to be a child without adult supervision; the execution control module is provided with a flag flag_c for indicating whether there is a child with or without adult supervision. When there is a child with or without adult supervision, flag_c=1; otherwise, flag_c=0; when flag_c=0 and flag_b=1, the execution control module controls the issuance of a related first voice reminder; when flag_c=1 and flag_b≠1, the execution control module controls the issuance of a related second voice reminder; when flag_c=1 and flag_b=1, the execution control module controls the issuance of a related third voice reminder.

[0014] Furthermore, the method by which the video processing module determines whether the human body is a child is:

[0015] A first calibration position and a second calibration position are set in the detection area, wherein the first calibration position is located at a side of the automatic door close to the camera, and the second calibration position is located at a side of the automatic door away from the camera;

[0016] Place the first upright pole at a first calibration position, and place the second upright pole at a second calibration position, wherein the heights of the first upright pole and the second upright pole are equal to a preset height threshold for determining whether the human body is a child;

[0017] Determine the ordinate y of the first calibration position in the video information 11 , the ordinate y of the second calibration position 21 , the vertical coordinate y of the top of the first pole image 12 and the vertical coordinate y of the top of the second pole image 22 ;

[0018] Calculate the height determination coefficient K, the calculation formula is as follows:

[0019] K=[(y 12 -y 11 )-(y 22 -y 21 )] / (y 21 -y 11 )

[0020] Determine the bottom y coordinate of the human body in the video information 31 and the top ordinate y 32 ;

[0021] Calculate the coordinate height h of the human body 1 and height determination threshold h 0 , the calculation formula is as follows:

[0022] h 1 =y 32 -y 31

[0023] h 0 =(y 32 -y 22 )×K+(y 22 -y 21 )

[0024] If h 1 ≤h 0 , then the human body is judged to be a child.

[0025] Furthermore, after identifying the human body, the video processing module also calculates the number of human bodies in the detection area. If the number of human bodies in the detection area reaches or exceeds a preset threshold number of people, it is determined that there is crowding in front of the automatic door. The execution control module is provided with a flag flag_d for indicating whether there is crowding. When there is crowding, flag_d=1; otherwise, flag_d=0; when flag_d=1, the execution control module controls the issuance of a related fourth voice reminder.

[0026] Furthermore, when the video processing module identifies a walking assistance tool among the objects in the detection area, it also determines whether the boundary box of the human body overlaps with the boundary box of the walking assistance tool. If so, it is determined that a person with limited mobility is about to pass through the automatic door; the execution control module is provided with a mark flag_e for indicating whether a person with limited mobility needs to pass through the automatic door. When a person with limited mobility is about to pass through the automatic door, flag_e=1; otherwise, flag_e=0; when flag_d=1, the execution control module controls the issuance of the relevant fifth voice reminder.

[0027] Furthermore, when the video processing module identifies a large piece of luggage among the objects in the detection area, it also determines whether the boundary box of the human body overlaps with the boundary box of the large piece of luggage. If so, it is determined that a person carrying the large piece of luggage is about to pass through the automatic door; the execution control module is provided with a flag flag_f for indicating whether a person carrying the large piece of luggage needs to pass through the automatic door. When a person carrying the large piece of luggage is about to pass through the automatic door, flag_f=1; otherwise, flag_f=0; when flag_f=1, the execution control module controls the issuance of the relevant sixth voice reminder.

[0028] Furthermore, when the video processing module identifies dangerous goods among the objects in the detection area, it also determines whether the boundary box of the human body overlaps with the boundary box of the dangerous goods. If there is overlap, it is determined that a person carrying dangerous goods is about to pass through the automatic door; the execution control module is provided with a flag flag_g for indicating whether a person carrying dangerous goods needs to pass through the automatic door. When a person carrying dangerous goods is about to pass through the automatic door, flag_g=1; otherwise, flag_g=0; when flag_g=1, the execution control module issues an alarm and issues the relevant seventh voice reminder.

[0029] In the present invention, by extracting video information near the automatic door from the existing system with cameras in public places such as shopping malls and office buildings, there is no need to set up a separate camera, which can save system costs; by dividing the detection area and the execution area, and using computer vision technology and deep learning model technology to detect the position of the human body, and then controlling the opening and closing of the automatic door, the detection blind area can be eliminated, and the control of the automatic door is more accurate and reasonable, thereby achieving integrated control of video monitoring and automatic door automatic control, making the automatic door management and control system safer, more efficient, energy-saving, and cost-saving. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 This is a structural block diagram of an embodiment of an automatic door control system based on machine vision of the present invention.

[0032] Figure 2 This is a structural block diagram of this embodiment when connected to a monitoring system.

[0033] Figure 3 A schematic top view of the division of the detection area and the execution area, and the setting of the first calibration position and the second calibration position.

[0034] Figure 4 for Figure 3 It is a side view.

[0035] Figure 5 It is a schematic diagram of a detection area, an execution area, a first calibration position and a second calibration position in video information.

[0036] The accompanying drawings in the specification are numeraled as follows:

[0037] Video access module-110; video processing module-120; execution control module-130; monitoring camera-210; monitoring module-220; automatic door-300; detection area-310; first calibration position-311; second calibration position-312; execution area-320; boundary box area-330. DETAILED DESCRIPTION

[0038] The following describes the implementation methods of the present invention through specific examples. The illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present invention. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0039] The existing automatic door control system generally senses the presence of a human body near the automatic door through infrared, microwave, ultrasonic, radar and other inductive sensors, and then outputs this sensing signal to the drive unit to control the motor action of the automatic door. However, this automatic door control system also generally has the following defects:

[0040] First, in terms of stability, since the detection senses whether there are objects in the vicinity of the automatic door, as long as the object is within the scanning range of the sensor, it will respond and send out a contact signal to start the motor. This method will also trigger the automatic door switch when some animals such as cats and dogs pass by. And these sensors are easily disturbed by external conditions, such as magnetic fields, temperature, etc., which can easily cause the automatic door switch to be triggered by mistake.

[0041] Secondly, in terms of safety, the perception methods of existing automatic door control systems not only have certain blind spots (for example, if you walk from the side of the door to the door, the system will not detect it), but are also designed for adults and often fail to detect children near the automatic door. Therefore, there are also major safety hazards.

[0042] Third, in terms of energy saving, the infrared sensor senses all objects in the sensing area, and pedestrians passing near the automatic door may also mistakenly trigger the automatic door to open, increasing power consumption. In view of the above problems, the present invention proposes a new automatic door control system based on machine vision.

[0043] See also Figure 1 , Figure 1 The structure block diagram of an embodiment of the automatic door control system based on machine vision of the present invention includes a video access module 110 , a video processing module 120 and an execution control module 130 .

[0044] The video access module 110 is used to extract video information from a camera or a system with a camera; the video acquisition module is a video splitter, and the video splitter is used to split a video information from the system with a camera and transmit it to the video processing module 120. Since a monitoring system is generally provided near the automatic door 300, the video access module 110 extracting video information from the monitoring system is used as an example for explanation in this embodiment.

[0045] See also Figure 2, the monitoring system includes a monitoring camera 210 and a monitoring module 220. Of course, there can be multiple monitoring cameras 210. The video splitter is used to divide the video information collected by the monitoring camera 210 near the automatic door 300 into two paths, one of which is still sent to the monitoring module 220, and the other is sent to the video processing module 120, so that the video information can be obtained by relying on the existing monitoring system without the need to set up a separate camera to obtain the video information. In addition, since the monitoring camera 210 generally has processing functions such as preliminary denoising and image enhancement, the accuracy of subsequent analysis can also be improved. Of course, the monitoring camera 210 needs to meet the requirements of accurately controlling the position of the automatic door 300. If it cannot be met, the position of the monitoring camera 210 can be adjusted so that the monitoring camera 210 can simultaneously meet the video monitoring requirements and the requirements of accurately controlling the position of the automatic door 300.

[0046] The video processing module 120 is used to define a detection area 310 in the video information, and identify objects in the detection area 310, and determine the position of a human body when the human body is identified. The execution control module 130 is used to define an execution area 320 in the middle of the detection area 310, and when the position of the human body identified by the video processing module 120 is located in the execution area 320, issue a control instruction to open the automatic door 300.

[0047] See also Figure 3 , Figure 4 and Figure 5 , the video processing module 120 defines the rectangular area near the automatic door 300 in the video information as the detection area 310, defines the rectangular area where the automatic door 300 is located as the execution area 320, and establishes a plane coordinate system in the video information, and calibrates the coordinate range of the detection area 310 and the coordinate range of the execution area 320. In this embodiment, a rectangular area is first defined near the automatic door 300 as the detection area 310, and a rectangular area is defined in the middle of the detection area 310 (i.e., the area closer to the automatic door 300) as the execution area 320, and a physical mark is made on the boundary between the detection area 310 and the execution area 320, and then the detection area 310 and the execution area 320 are defined in the video information according to the physical mark image. Of course, the detection area 310 and the execution area 320 only need to be delineated once at the beginning. Afterwards, the coordinate ranges of the detection area 310 and the execution area 320 can be stored in the video processing module 120. When used subsequently, the coordinate ranges of the detection area 310 and the execution area 320 can be directly retrieved from the video processing module 120.

[0048] Since both the detection area 310 and the execution area 320 are rectangular areas, when calibrating the coordinate range of the detection area 310 and the execution area 320 in the video information, only the coordinates of the two farthest corner points of the detection area 310 and the execution area 320 in the video information can be input. For example, in this embodiment, the coordinates A of the upper left corner point of the detection area 310 in the video information are used. 1 (x a1 ,y a1 ) and the coordinates of the lower right corner point B 1 (x b1 ,y b1 ) represents the range of the detection area 310, and the coordinate A of the upper left corner of the execution area 320 in the video information 2 (x a2 ,y a2 ) and the coordinates of the lower right corner point B 2 (x b2 ,y b2 ) represents the range of the execution area 320.

[0049] The video processing module 120 generally uses a target detection algorithm to identify objects (including human bodies). Since only objects within the detection area 310 are identified in this embodiment, each frame of the video information can be compared with the calibrated coordinates of the detection area 310 for cropping, and image information outside the detection area 310 can be removed. Only the detection area 310 is identified, which can greatly reduce the amount of calculation of the algorithm and improve calculation efficiency.

[0050] The target detection algorithm can use the YOLO (You Only Look Once) algorithm or the SSD (Single Shot MultiBox Detector) algorithm. Both the YOLO algorithm and the SSD algorithm are efficient real-time target detection algorithms and are widely used in human detection. YOLO achieves fast detection by dividing the entire image into grids and predicting bounding boxes and categories in each grid, while SSD performs detection on feature maps of different scales to improve accuracy. To accurately locate the position and posture of the human body, a posture estimation algorithm (such as PoseNet, OpenPose, or HRNet) can be further applied on the basis of target detection, which can provide key points and posture information of the human body.

[0051] In this embodiment, each cropped frame is sent to the target detection algorithm YOLO5 to identify whether the object in the image is a human body or other objects (such as a suitcase, a cane, a wheelchair, a knife, etc.), and each human body or other object detected in the image is given its accurate coordinates and bounding box area 330. Each bounding box area 330 can also be represented by its upper left corner coordinate A 3 (x a3 ,y a3) and the coordinates of the lower right corner point B 3 (x b3 ,y b3 ) indicates its range.

[0052] After identifying the human body, the video processing module 120 first identifies the bounding box area 330 of the human body, and then accurately identifies the feet of the human body in the bounding box area 330. For example, the bounding box area 330 can be sent to the PoseNet posture estimation algorithm to accurately identify the hands and feet of the human body. Afterwards, the coordinates of the feet are obtained. When the coordinates of at least one foot of the human body are located within the coordinate range of the execution area 320 in a predetermined number of consecutive frames (the specific number of frames is based on accurate identification without misjudgment), it is determined that the position of the human body is located in the execution area 320. Since there will be errors in determining single-frame data, this embodiment uses a method of continuously determining whether the foot positions of the same person in multiple frames of data are all in the execution area 320 domain to better eliminate errors.

[0053] The execution control module 130 is provided with a flag flag_a for indicating whether the position of the human body is located in the execution area 320 and a flag flag_b for indicating whether the person has stayed in the execution area 320 for a long time. When the position of the human body is located in the execution area 320, flag_a=1; when the position of the human body is not located in the execution area 320, flag_a=0. When flag_a=1, it means that the human body is very close to the switch area of ​​the automatic door 300, and it can be determined that the human body wants to pass through the automatic door 300, so the execution control module 130 issues a control instruction to open the automatic door 300 to open the automatic door 300; until flag_a=0, the execution control module 130 issues a control instruction to close the automatic door 300 to close the automatic door 300.

[0054] When it is detected that a human body is located in the execution area 320, the execution control module 130 also counts the time the human body stays in the execution area 320. When the timing time of any human body reaches the predetermined time threshold, the execution control module 130 sets flag_b=1. If the position of the human body leaves the execution area 320 before the timing time reaches the predetermined time threshold, the timing of the human body is stopped and the timing time of the human body is restored to the initial value. When flag_b=1 and another flag flag_c=0, the execution control module 130 controls the issuance of the relevant first voice reminder (such as: please do not stay by the door, please pass quickly, etc.). At this time, the execution control module 130 can also send a command to the monitoring module 220 to enable the monitoring module 220 to intercept this video and upload it to the cloud system. And the timing time is restored to the initial value and the timing is restarted.

[0055] After identifying the human body, the video processing module 120 further determines whether the human body is a child. The method for the video processing module 120 to determine whether the human body is a child includes the following steps:

[0056] A first calibration position 311 and a second calibration position 312 are pre-set in the detection area 310. The first calibration position 311 is located on the side of the automatic door 300 close to the camera, and the second calibration position 312 is located on the side of the automatic door 300 far from the camera. In this embodiment, the camera is set on the perpendicular bisector of the automatic door 300, and the intersection of the two intersections of the contour line of the detection area 310 and the perpendicular bisector of the automatic door 300, which is close to the camera, is the first calibration position 311, and the other intersection is the second calibration position 312, so as to reduce calculation errors.

[0057] The first pole is placed at the first calibration position 311, and the second pole is placed at the second calibration position 312. The heights of the first pole and the second pole are equal to a preset height threshold for determining whether a person is a child. For example, when 1.2 meters is used as the height threshold for determining whether a person is a child (i.e., a person is determined to be a child if his / her height is not higher than 1.2 meters), the heights of the first pole and the second pole are also 1.2 meters.

[0058] Determine the ordinate y of the first calibration position 311 in the video information 11 , the ordinate y of the second calibration position 312 21 , the vertical coordinate y of the top of the first pole image 12 and the vertical coordinate y of the top of the second pole image 22 .

[0059] Calculate the height determination coefficient K, the calculation formula is as follows:

[0060] K=[(y 12 -y 11 )-(y 22 -y 21 )] / (y 21 -y 11 )

[0061] Then, the height determination coefficient K, the ordinate y of the second calibration position 21 and the vertical coordinate y of the top of the second pole image 22 The data is stored in the video processing module 120 for use in subsequent judgment.

[0062] When determining whether the human body in the video information is a child, first determine the bottom vertical coordinate y of the human body in the video information 31 (i.e., the ordinate y in the coordinates of the upper left corner point of the bounding box area 330 of the human body a3 ) and the top ordinate y 32(i.e., the ordinate y in the coordinates of the lower right corner point of the human body bounding box area 330 b3 ), and then calculate the coordinate height h of the human body 1 and height determination threshold h 0 , the calculation formula is as follows:

[0063] h 1 =y 32 -y 31

[0064] h 0 =(y 32 -y 22 )×K+(y 22 -y 21 )

[0065] If h 1 ≤h 0 , then the human body is judged to be a child.

[0066] When the human body is a child, it is further detected whether there is an adult within the preset distance threshold range. If there is no adult, the child is judged to be a child without adult supervision. The execution control module 130 is provided with a flag flag_c for indicating whether there is a child with or without adult supervision. When there is a child with or without adult supervision, flag_c=1; otherwise, flag_c=0; when flag_c=1, and flag_b≠1, the execution control module 130 controls the issuance of a related second voice reminder (such as: please ask adults to take good care of their children, etc.). At this time, the execution control module 130 can also send a command to the monitoring module 220, so that the monitoring module 220 intercepts this video and uploads it to the cloud system. When flag_c=1, and flag_b=1, it will be judged that the child is playing near the automatic door 300, and the execution control module 130 controls the issuance of a related third voice reminder (such as: please do not play and stay near the door, please ask adults to take good care of your children, etc.). At this time, the execution control module 130 may also send a command to the monitoring module 220 to enable the monitoring module 220 to capture this video segment and upload it to the cloud system.

[0067] After identifying the human body, the video processing module 120 also calculates the number of human bodies in the detection area 310. If the number of human bodies in the detection area 310 reaches or exceeds the preset threshold, it is determined that there is crowding in front of the automatic door 300. The execution control module 130 is provided with a flag flag_d for indicating whether there is crowding. When there is crowding, flag_d=1; otherwise, flag_d=0; when flag_d=1, the execution control module 130 controls the issuance of the relevant fourth voice reminder (such as: there are many pedestrians at present, please do not crowd, pass in an orderly manner, etc.). At this time, the execution control module 130 can also send a command to the monitoring module 220, so that the monitoring module 220 intercepts this video and uploads it to the cloud system.

[0068] In order to facilitate timely reminders for different situations, the video processing module 120 will also identify objects such as walking assistance tools, large luggage, and dangerous goods. When the video processing module 120 identifies walking assistance tools (such as crutches or wheelchairs) in the objects in the detection area 310, it also determines whether the boundary box of the human body overlaps with the boundary box of the walking assistance tool. If there is an overlap, it is determined that a person with limited mobility is about to pass through the automatic door 300; the execution control module 130 is provided with a flag flag_e for indicating whether a person with limited mobility needs to pass through the automatic door 300. When a person with limited mobility is about to pass through the automatic door 300, flag_e=1; otherwise, flag_e=0; when flag_d=1, the execution control module 130 controls the issuance of the relevant fifth voice reminder (such as: there are pedestrians with limited mobility passing, please pay attention to safety, and the companions should pay attention to care, etc.). At this time, the execution control module 130 can also send a command to the monitoring module 220, so that the monitoring module 220 intercepts this video and uploads it to the cloud system.

[0069] When the video processing module 120 identifies a large piece of luggage (such as a large suitcase) among the objects in the detection area 310, it also determines whether the boundary box of the human body overlaps with the boundary box of the large piece of luggage. If so, it is determined that a person carrying large luggage is about to pass through the automatic door 300; the execution control module 130 is provided with a flag flag_f for indicating whether a person carrying large luggage needs to pass through the automatic door 300. When a person carrying large luggage is about to pass through the automatic door 300, flag_f=1; otherwise, flag_f=0; when flag_f=1, the execution control module 130 controls the issuance of the relevant sixth voice reminder (such as: please pay attention to safety when carrying large luggage, pass quickly, etc.). At this time, the execution control module 130 can also send a command to the monitoring module 220, so that the monitoring module 220 intercepts this video and uploads it to the cloud system.

[0070] When the video processing module 120 identifies dangerous objects (such as knives, etc.) in the objects in the detection area 310, it also determines whether the boundary box of the human body overlaps with the boundary box of the dangerous object. If there is overlap, it is determined that a person carrying dangerous objects is about to pass through the automatic door 300; the execution control module 130 is provided with a flag flag_g for indicating whether a person carrying dangerous objects needs to pass through the automatic door 300. When a person carrying dangerous objects is about to pass through the automatic door 300, flag_g=1; otherwise, flag_g=0; when flag_g=1, the execution control module 130 controls the alarm to sound and controls the issuance of the relevant seventh voice reminder (such as: Do not bring dangerous objects in, etc.). At this time, the execution control module 130 can also send a command to the monitoring module 220, so that the monitoring module 220 intercepts this video and uploads it to the cloud system.

[0071] By adopting the above-mentioned identification and reminder methods, security personnel can understand the situation in front of the door and take corresponding disposal measures in time.

[0072] Since the monitoring system has been connected in this embodiment, the monitoring module 220 can also use the deep learning model to determine from the video information whether there are pre-trained specific events (such as failure to open the automatic door 300, carrying prohibited items in and out, etc.). If so, the execution control module 130 selects the automatic door 300 control execution plan corresponding to the specific event, thereby increasing the ability to handle special situations.

[0073] The existing automatic door sensor devices have problems such as less information, blind spots, inability to detect safety hazards such as the elderly and children, and inability to adapt to the requirements of modern buildings for safety, stability, and intelligence. At the same time, the existing traditional automatic door sensor devices cannot take into account safety, environmental comfort, energy saving, and efficiency at the same time. In order to provide the safety, stability, and intelligence of the automatic door 300, the usual solution is to directly integrate a camera and processor dedicated to the automatic door control system at the top or near the door. This camera and processor greatly increase the cost of the traditional automatic door 300, so this solution has not been widely used.

[0074] In this embodiment, by combining with the existing video surveillance systems in public places such as shopping malls and office buildings, video information near the automatic door 300 is sampled and obtained, and there is no need to set up a separate camera, which can save system costs; by dividing the detection area 310 and the execution area 320, and using computer vision technology and deep learning model technology to detect the position of the human body, and then controlling the opening and closing of the automatic door 300, the detection blind spot can be eliminated, and the control of the automatic door 300 is more accurate and reasonable, thereby achieving integrated control of video surveillance and automatic control of the automatic door 300, making the automatic door management and control system safer, more efficient, energy-saving, and cost-saving.

[0075] The above embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An automatic door control system based on machine vision, characterized in that: include A video access module, used for extracting video information from a camera or a system having a camera; The video processing module is used to define a detection area in the video information, identify objects in the detection area, and determine the position of a human body when it is identified; as well as The execution control module is used to define an execution area in the middle of the detection area, and when the position of the human body identified by the video processing module is located in the execution area, issue a control instruction to open the automatic door.

2. The automatic door control system based on machine vision as claimed in claim 1, characterized in that: The video acquisition module is a video splitter, which is used to split a video information from a system with a camera and transmit it to the video processing module.

3. The automatic door control system based on machine vision as claimed in claim 1, characterized in that: The video processing module defines a rectangular area near the automatic door in the video information as a detection area, defines a rectangular area where the automatic door is located as an execution area, and establishes a plane coordinate system in the video information to calibrate the coordinate range of the detection area and the coordinate range of the execution area respectively; After identifying the human body, the video processing module first identifies the bounding box area of ​​the human body, then accurately identifies the feet of the human body in the bounding box area, and obtains the coordinates of the feet. When the coordinates of at least one foot of the human body are within the coordinate range of the execution area in a predetermined number of consecutive frames, it is determined that the position of the human body is located in the execution area.

4. The automatic door control system based on machine vision as claimed in claim 3, characterized in that: The execution control module is provided with a mark flag_a for indicating whether the position of a human body is located in the execution area. When it is detected that the position of a human body is located in the execution area, flag_a=1; otherwise, flag_a=0; when flag_a=1, the execution control module issues a control instruction to open the automatic door, until flag_a=0, the execution control module issues a control instruction to close the automatic door; the execution control module is also provided with a mark flag_b for indicating whether a person stays in the execution area for a long time. When it is detected that the position of a human body is located in the execution area, the time that the person stays in the execution area is also counted. When the timing time reaches a predetermined time threshold, the execution control module sets flag_b=1; if the position of the person leaves the execution area before the timing time reaches the predetermined time threshold, the timing of the person is stopped and the timing time of the person is restored to the initial value.

5. The automatic door control system based on machine vision as claimed in claim 4, characterized in that: After identifying the human body, the video processing module also determines whether the human body is a child. When the human body is a child, it further detects whether there is an adult within a preset distance threshold range. If there is no adult, the child is determined to be a child without adult supervision; the execution control module is provided with a flag flag_c for indicating whether there is a child without adult supervision. When there is a child without adult supervision, flag_c=1; otherwise, flag_c=0; when flag_c=0 and flag_b=1, the execution control module controls the issuance of a related first voice reminder; when flag_c=1 and flag_b≠1, the execution control module controls the issuance of a related second voice reminder; when flag_c=1 and flag_b=1, the execution control module controls the issuance of a related third voice reminder.

6. The automatic door control system based on machine vision as claimed in claim 5, characterized in that: The method for the video processing module to determine whether the human body is a child is: A first calibration position and a second calibration position are set in the detection area, wherein the first calibration position is located at a side of the automatic door close to the camera, and the second calibration position is located at a side of the automatic door away from the camera; Place the first upright pole at a first calibration position, and place the second upright pole at a second calibration position, wherein the heights of the first upright pole and the second upright pole are equal to a preset height threshold for determining whether the human body is a child; Determine the ordinate y of the first calibration position in the video information 11 , the ordinate y of the second calibration position 21 , the vertical coordinate y of the top of the first pole image 12 and the vertical coordinate y of the top of the second pole image 22 ; Calculate the height determination coefficient K, the calculation formula is as follows: K=[(and 12 -and 11 )-(and 22 -and 21 )] / (and 21 -and 11 ) Determine the bottom y coordinate of the human body in the video information 31 and the top ordinate y 32 ; Calculate the coordinate height h1 and height determination threshold h0 of the human body. The calculation formula is as follows: h1=y 32 -and 31 h0=(y 32 -y 22 )×K+(y 22 -y 21 ) If h1≤h0, the human body is judged to be a child.

7. The automatic door control system based on machine vision according to any one of claims 1 to 6, characterized in that: After identifying the human body, the video processing module also calculates the number of human bodies in the detection area. If the number of human bodies in the detection area reaches or exceeds a preset threshold number of people, it is determined that there is crowding in front of the automatic door. The execution control module is provided with a flag flag_d for indicating whether there is crowding. When there is crowding, flag_d=1; otherwise, flag_d=0; when flag_d=1, the execution control module controls the issuance of a related fourth voice reminder.

8. The automatic door control system based on machine vision according to any one of claims 1 to 6, characterized in that: When the video processing module identifies the walking assistance tool among the objects in the detection area, it also determines whether the boundary box of the human body overlaps with the boundary box of the walking assistance tool. If so, it is determined that a person with limited mobility is about to pass through the automatic door. The execution control module is provided with a flag flag_e for indicating whether a person with limited mobility needs to pass through the automatic door. When a person with limited mobility is about to pass through the automatic door, flag_e=1. Otherwise, flag_e=0; when flag_d=1, the execution control module controls to issue the relevant fifth voice reminder.

9. The automatic door control system based on machine vision according to any one of claims 1 to 6, characterized in that: When the video processing module identifies the large luggage among the objects in the detection area, it also determines whether the boundary box of the human body overlaps with the boundary box of the large luggage. If so, it is determined that a person carrying the large luggage is about to pass through the automatic door; the execution control module is provided with a flag flag_f for indicating whether a person carrying the large luggage needs to pass through the automatic door. When a person carrying the large luggage is about to pass through the automatic door, flag_f=1; otherwise, flag_f=0; when flag_f=1, the execution control module controls the issuance of the relevant sixth voice reminder.

10. The automatic door control system based on machine vision according to any one of claims 1 to 6, characterized in that: When the video processing module identifies dangerous goods among the objects in the detection area, it also determines whether the boundary box of the human body overlaps with the boundary box of the dangerous goods. If there is overlap, it is determined that a person carrying dangerous goods is about to pass through the automatic door; the execution control module is provided with a flag flag_g for indicating whether a person carrying dangerous goods needs to pass through the automatic door. When a person carrying dangerous goods is about to pass through the automatic door, flag_g=1; otherwise, flag_g=0; when flag_g=1, the execution control module sounds an alarm and issues the relevant seventh voice reminder.

Images