Monitoring terminal for children to independently take elevator based on video analysis technology

Abstract

The invention discloses a video analysis technology-based monitoring terminal for a child to take an elevator alone, relates to the technical field of elevator monitoring, and solves the problem that whether the child takes the elevator alone or not cannot be effectively identified. According to the method, the moving object can be accurately determined from the video of the elevator door, the moving object can be accurately recognized through calculation and comparison of pixel point features, gradient values and the like and determination of the same point location and the moving point location, the object recognition accuracy in a complex video scene is improved, and the user experience is improved. According to the method, the moving object and the related human body template are combined and compared to recognize the character objects and determine the specific number of the character objects entering the elevator, the height of the character objects is determined through the height determining end based on feature point mapping and distance calculation, whether the person entering the elevator is a child or not can be accurately judged, and the safety of the elevator is improved. And a reliable basis is provided for subsequent early warning.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator monitoring, and in particular to a monitoring terminal for infants riding alone in an elevator based on video analysis technology. Background Art

[0002] Elevator monitoring is crucial to ensure passenger safety and the normal operation of elevators. In modern buildings, elevators, as a key tool for vertical transportation, carry a large number of people every day, and there may be various potential risks, such as the dangers that young children may face when riding alone, and people may be trapped due to elevator failures. Effective elevator monitoring can detect abnormal situations in a timely manner, prevent accidents, and provide users with a safer environment.

[0003] The application with publication number CN103508282A discloses an elevator monitoring device and a monitoring method. When a load sensor and an image sensor are combined to determine whether there is someone in the elevator car, the underreporting of extremely light and immeasurable persons such as infants or small animals can be suppressed, and the burden of image processing performed by the image sensor can be reduced; the image of the camera unit is processed to detect new passengers entering the car, and whether the new passenger is a specified light passenger, i.e., an immeasurable person, is determined based on the change in the load value of the load sensor when the new passenger boards. If the new passenger is determined to be an immeasurable person, image tracking is performed to determine whether the immeasurable person has mixed into the car until the immeasurable person exits the car. If it is determined that the immeasurable person has mixed into the car, it is determined as "someone".

[0004] Regarding the current monitoring of people inside elevators, the number of people inside is identified based on the load of the corresponding elevator, so as to assess whether the elevator has safety problems such as overloading. However, the original monitoring method of this type of elevator only focuses on overweight situations and cannot effectively identify whether there are young children riding the elevator alone. The original elevator monitoring method is not very targeted in terms of safety assurance and does not provide safety assurance for people such as young children. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention provides a monitoring terminal for young children riding alone in an elevator based on video analysis technology, which solves the problem of being unable to effectively identify whether there is a young child riding alone in the elevator.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A monitoring terminal for children riding elevators alone based on video analysis technology, comprising:

[0007] The door monitoring terminal monitors the elevator door. When the elevator door is opened, the opening time period is recorded and the recorded opening time period is transmitted to the video analysis and processing terminal.

[0008] The real-time video terminal is set at the central high point of the elevator floor, monitors the elevator door in real time, and transmits the real-time monitored video to the video analysis and processing terminal;

[0009] The video analysis and processing end determines the video segments belonging to the open time period from the real-time monitored video based on the recorded open time period, performs frame analysis on the determined video segments, determines the moving points between adjacent frames, and then determines the moving objects based on the features of the corresponding moving points. The specific method is as follows:

[0010] Based on the video segment associated with the start time period, starting from the initial moment of the video segment, confirm the initial frame, then confirm the subsequent frames that appear in the video segment in sequence, and generate a frame sequence according to the time sequence;

[0011] Based on the determined frame sorting sequence, the characteristics of the pixels in each frame are confirmed, and the vertical gradient value T1 and the horizontal gradient value T2 of the corresponding pixel are preferentially determined. Confirm the feature Tz of the corresponding pixel point, and mark the pixel points with the same clustering features in adjacent frames as the same point position. The same clustering features include: the feature Tz confirmed by the corresponding pixel point is consistent, and the feature Tz associated with the surrounding pixel points with the same position feature of the corresponding pixel point is also consistent. Further:

[0012] In the previous set of frames: take the corresponding pixel point as the starting point, confirm the location of the surrounding pixel points and use it as the end point, and confirm a set of vectors based on the starting point and the end point;

[0013] In the next set of frames: the same method is used to confirm the vector, the surrounding pixels with the same vector features are marked as the surrounding pixels with the same position features, and the feature Tz consistency comparison is performed;

[0014] Based on the same points marked in adjacent frames, the same points in different positions are marked as moving points: the corresponding frames are considered to be a set of planes, and the same points have position coordinates in the planes. When the position coordinates of the points in the next set of frames change, the same points are classified as moving points. Based on the moving points confirmed in succession in several frames in the frame sorting sequence, the objects covered by several adjacent moving points are marked as moving objects.

[0015] The moving object confirmation end identifies the frame corresponding to the moving object area ratio from a number of frame images, and marks the moving object appearing in this frame image as the object to be checked, and then checks the determined object to be checked with the human body contour template and determines the human object, and then determines whether to execute the height determination end based on the number of human objects. The specific method is as follows:

[0016] Confirm that the area ratio Zb of the corresponding moving object in each group of frames is Zb = the total area of ​​the moving object ÷ the total area of ​​the frames, mark the frame with the maximum area ratio Zb as the frame to be checked, mark the moving object in the frame to be checked as the object to be checked, determine the center point of the object to be checked, align the center point of the object to be checked with the center point of the human body contour template, the center point of the human body contour template is the preset point, and then the human body contour template is scaled proportionally according to the center point inside it, and the scaling is stopped when the overlap area of ​​the object to be checked and the human body contour template reaches the maximum state. During the scaling process, the object to be checked is located in the human body contour template, and it is confirmed that the overlap area is located in the coverage ratio of the human body contour template, and the coverage ratio = the overlap area ÷ the total area of ​​the human body contour template. When the coverage ratio is ≥80%, the object to be checked is marked as a human object. When the coverage ratio is <80%, no specific calibration of the human object is performed;

[0017] Identify the number of human objects in this video segment. If the number is 1, execute the moving object confirmation end. If the number is ≥ 2, do not perform any processing.

[0018] The height determination end, based on the determined human object, locks the highest feature point of the human object in the human body outline template, and then confirms the feature base points associated with the two sets of feature points mapped on the same plane based on the feature points at the real-time video end, and confirms the spatial position of the highest feature point based on the distance between the feature base points, thereby determining the specific height of the human object, and based on the determination result, locking whether the human object is a child, the specific method is as follows:

[0019] Based on the highest point preset in the human body contour template, the contour point closest to the highest point is identified from the human object, and the identified contour point is calibrated as the highest feature point. Then, based on the lowest points preset on the left and right sides of the human body contour template, the point closest to the lowest point is locked from the edge contour of the human object and calibrated as the lowest feature point. There are two groups of lowest feature points. The two groups of lowest feature points are connected to confirm a group of feature connection lines, and a vertical line A vertically downward from the highest feature point is constructed. The intersection of the vertical line A and the feature connection line is calibrated as the mapping base point A of the highest feature point.

[0020] Based on the feature points of the real-time video recording end, confirm the vertical line B pointing vertically downward, and then determine the intersection of the vertical line B and the elevator edge line and mark it as the mapping base point B;

[0021] Directly connect the mapping base point A and the mapping base point B, confirm the base point connection line, move the vertical line A according to the confirmed base point connection line, make the vertical line A coincide with the vertical line B, and confirm the virtual length C1 of the vertical line A and the virtual length C2 of the vertical line B, and use GD=C1÷C2 to confirm the height ratio value GD of the human object;

[0022] The confirmed height proportion value GD is compared with the preset value Y1, where Y1 is the preset value. When GD≤Y1, the person object is calibrated as a child and a child signal is generated and transmitted to the monitoring and early warning center. When GD>Y1, no calibration is performed.

[0023] Preferably, it also includes:

[0024] The monitoring and early warning center issues real-time early warnings based on the child signals sent by the designated elevator, and conducts real-time video monitoring of the elevator while maintaining voice call functions.

[0025] The present invention provides a monitoring terminal for children riding alone in an elevator based on video analysis technology. Compared with the prior art, it has the following beneficial effects:

[0026] The present invention can accurately determine the moving objects from the video of the elevator door through a detailed frame picture analysis method, and can accurately identify the moving objects through the calculation and comparison of pixel point features, gradient values, etc., as well as the determination of the same point position and the moving point position, thereby improving the accuracy of object recognition in complex video scenes, and combining and comparing the moving objects with the relevant human body templates to identify the human objects, and determine the specific number of human objects entering the elevator;

[0027] By determining the height of the person object based on feature point mapping and distance calculation, the height determination end can more accurately judge whether the person entering the elevator is a child, providing a reliable basis for subsequent warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the principle framework of the present invention;

[0029] Figure 2 This is a schematic diagram of determining the height of a human object according to the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] See also Figure 1 The present application provides a monitoring terminal for a child riding an elevator alone based on video analysis technology, including a door monitoring terminal, a real-time video recording terminal, a video analysis and processing terminal, a moving object confirmation terminal, a height determination terminal, and a monitoring and early warning center, wherein the real-time video recording terminal, the video analysis and processing terminal, the moving object confirmation terminal, the height determination terminal, and the monitoring and early warning center are electrically connected from an output node to an input node in sequence, and the door monitoring terminal is electrically connected to the video analysis and processing terminal input node;

[0032] The door monitoring end monitors the elevator door. When the elevator door is opened, the opening time period is recorded and the recorded opening time period is transmitted to the video analysis and processing end.

[0033] The real-time video terminal is set at the central high point of the elevator bottom plate (such as Figure 2 The first feature point corresponding to the real-time video terminal is the installation position of the real-time video terminal), the elevator door is monitored in real time, and the real-time monitored video is transmitted to the video analysis and processing terminal (when the elevator door is opened, the video monitored by the real-time video terminal is the relevant picture outside the elevator. When a corresponding person enters the elevator, it can also be monitored in real time);

[0034] Among them, the video analysis processing end determines the video segment belonging to the open time period from the real-time monitored video based on the recorded open time period, and performs frame analysis on the determined video segment, determines the moving point between adjacent frames, and then determines the moving object (which may be a person or other animals, etc.) based on the characteristics of the corresponding moving point. Specifically, since the object entering the elevator may be an adult, a child or other small animals, etc., in order to better distinguish them, it is necessary to first confirm the corresponding moving object, and then check the confirmed moving object with the relevant template to identify whether it is the corresponding person, so as to determine whether it is a single person. If it is a single person, it is necessary to execute subsequent related analysis processes to assess whether the single person is a child, so as to carry out relevant monitoring processes on the child to ensure the safety of the corresponding child riding the elevator alone;

[0035] Among them, for the video analysis and processing end, the specific method of determining the moving object based on the characteristics of the corresponding moving point is:

[0036] Based on the video segment associated with its open time period, starting from the initial moment of the video segment, confirm its initial frame, then confirm the subsequent frames that appear in the video segment in sequence, and generate a frame sequence according to the time sequence (the frame rate corresponding to each frame in the video segment is determined by the relevant recording equipment, but the relevant frame rate must not be lower than 24fps, otherwise it will be difficult to determine the moving point from the frame);

[0037] Based on the determined frame sorting sequence, the characteristics of the pixels in each frame are confirmed, and the vertical gradient value T1 and the horizontal gradient value T2 of the corresponding pixel are preferentially determined. Confirm the feature Tz of the corresponding pixel point, and mark the pixel points with the same clustering feature in the adjacent frame as the same point position. The same clustering feature includes: the feature Tz confirmed by the corresponding pixel point is consistent, and the feature Tz associated with several adjacent pixel points around this pixel point is also consistent, that is: if there are two point features A and B that are consistent, then it is necessary to identify the adjacent pixel points around point A, and then identify the adjacent pixel points around point B, and compare the features of the adjacent pixel points to identify whether they are consistent. The comparison is the comparison of the same position features, and it cannot be compared arbitrarily. The so-called same position features can be determined by determining the position features in the way of determining the vector. Take point A as the starting point and any point of the surrounding pixels as the end point, determine the vector, and then confirm the vector in point B and its surrounding pixels. The points with the same vector features are regarded as the same position feature points, and then perform feature comparison synchronously to evaluate whether they are consistent;

[0038] Based on the same points marked in adjacent frames, the same points in different positions are marked as moving points: the corresponding frames are assumed to be a set of planes, and the same points have position features (two-dimensional coordinates) in this plane. When the position features of this point change in the next set of frames (that is, the two-dimensional coordinates change accordingly), then this type of same point is a moving point. Based on the moving points confirmed in succession in several frames in the frame sorting sequence, the objects covered by several adjacent moving points are marked as moving objects (it can also be understood that a moving object is composed of several moving points. When there is only one set of moving objects, the generated moving points are all adjacent, so the covered objects can be confirmed. If there are non-adjacent moving points, it means that multiple moving objects have appeared).

[0039] The moving object confirmation end identifies the frame corresponding to the moving object area ratio from a number of frame images, and marks the moving object appearing in the frame image as the object to be checked, then checks the determined object to be checked with the human body contour template and determines the human object, and then determines whether to execute the height determination end based on the number of human objects;

[0040] The calibrated moving object is checked with the preset human body contour template, including: confirming the area ratio Zb of the corresponding moving object in each group of frames, where Zb=total area of ​​the moving object ÷ total area of ​​the frame (the total area of ​​each frame is consistent), calibrating the frame with the maximum area ratio Zb as the to-be-checked frame, calibrating the moving object in the to-be-checked frame as the to-be-checked object, determining the center point of the to-be-checked object (the center point of the object can be determined by the mean coordinate method, confirming the contour points of the to-be-checked object and placing them in the same two-dimensional coordinate system, averaging the point coordinates associated with several contour points, and confirming their mean coordinates, which are the center of the corresponding object. point), align the center point of the object to be checked with the center point of the human body contour template, the center point of the human body contour template is the preset point (prepared in advance by relevant personnel), and then the human body contour template is scaled proportionally according to the center point inside it, and the scaling is stopped when the overlap area between the object to be checked and the human body contour template reaches the maximum state. During the scaling process, the object to be checked is located in the human body contour template, and it is confirmed that the overlap area is located in the coverage ratio of the human body contour template, and the coverage ratio = overlap area ÷ total area of ​​the human body contour template. When the coverage ratio is ≥80%, the object to be checked is calibrated as a human object. Otherwise, no specific calibration of the human object is performed. The human body contour template is a preset template with a preset center point inside it;

[0041] Identify the number of human objects in this video segment. If the number = 1, execute the moving object confirmation terminal. If the number ≥ 2, do not perform any processing. When there is only one person, it means that there is only one person entering the elevator at this moment. If this person is a child, a safety warning is required. Therefore, it is necessary to perform height confirmation through the subsequent height confirmation terminal to identify whether the person entering the elevator is a child.

[0042] The height determination end, based on the determined human object, locks the highest feature point of the human object in the human body outline template, and then based on the feature points of the real-time video recording end, confirms the feature base points associated with the two sets of feature points mapped on the same plane, and confirms the spatial position of the highest feature point based on the distance between the feature base points, so as to determine the specific height of the human object, and locks whether the human object is a child based on the determination result. The overall performance area of ​​the human object here is the largest state (that is, the associated state of the corresponding object to be checked). Then, only when the human object is in a standing state with all four limbs spread out, will the area occupying the largest proportion of the frame screen appear. Therefore, the state of such an object to be checked at this moment must be a standing state. Because it needs to enter the elevator, the height of the highest point currently confirmed is the height of the corresponding person;

[0043] Combination Figure 2, based on the highest point preset in the human body contour template, identify the contour point closest to the highest point from the human object, and mark the identified contour point as the highest feature point (that is, the highest point corresponding to the human object, at this moment the human object is aligned with the center point of the human body contour template, and the human object is located in the human body contour template), and then based on the lowest points preset on the left and right sides of the human body contour template, lock the point closest to the lowest point from the edge contour of the human object and mark this point as the lowest feature point. There are two groups of lowest feature points. Connect the two groups of lowest feature points, confirm a group of feature lines, construct a vertical line A vertically downward from the highest feature point, and mark the intersection of the vertical line A and the feature line as the mapping base point A of the highest feature point;

[0044] Based on the feature points of the real-time video recording end, confirm the vertical line B pointing vertically downward, and then determine the intersection of the vertical line B and the elevator edge line and mark it as the mapping base point B. Because the internal environment of the elevator can be directly confirmed based on the associated model, the mapping base point B can also be directly preset in the corresponding model and can be used directly, so the corresponding mapping base point B can be directly locked;

[0045] Directly connect the mapping base point A and the mapping base point B, confirm the base point connection, move the vertical line A according to the confirmed base point connection, so that the vertical line A coincides with the vertical line B, and confirm the virtual length C1 of the vertical line A and the virtual length C2 of the vertical line B. The virtual length is the length parameter given by the system (not the real parameter), and GD=C1÷C2 is used to confirm the height ratio GD of the character object;

[0046] The confirmed height proportion value GD is compared with the preset value Y1, where Y1 is the preset value. Its specific value is confirmed based on the actual height of the elevator. Its general value is 0.5. Because the minimum height of the elevator is generally 2.3 meters and the height of a child is 1.2 meters, the value of 0.5 is more appropriate. When GD≤Y1, the person object is calibrated as a child and a child signal is generated and transmitted to the monitoring and early warning center. When GD>Y1, no calibration is performed.

[0047] Its monitoring and early warning center issues real-time early warnings based on the child signals sent by designated elevators, and conducts real-time video monitoring of the elevator and maintains the voice call function. Its relevant security personnel provide safety notices and turn off the early warning or corresponding voice call process in real time.

[0048] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0049] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A monitoring terminal for children riding alone in an elevator based on video analysis technology, characterized in that: include: The door monitoring terminal monitors the elevator door. When the elevator door is opened, the opening time period is recorded and the recorded opening time period is transmitted to the video analysis and processing terminal. The real-time video terminal is set at the central high point of the elevator floor, monitors the elevator door in real time, and transmits the real-time monitored video to the video analysis and processing terminal; The video analysis and processing end determines the video segment belonging to the open time period from the real-time monitored video based on the recorded open time period, performs frame analysis on the determined video segment, determines the moving points between adjacent frames, and then determines the moving object based on the features of the corresponding moving points; The moving object confirmation end identifies the frame corresponding to the moving object area ratio from a number of frame images, and marks the moving object appearing in the frame image as the object to be checked, then checks the determined object to be checked with the human body contour template and determines the human object, and then determines whether to execute the height determination end based on the number of human objects; The height determination end, based on the determined human object, locks the highest feature point of the human object in the human body contour template, and then based on the feature points at the real-time video recording end, confirms the feature base points associated with the two sets of feature points mapped on the same plane, and confirms the spatial position of the highest feature point based on the distance between the feature base points, so as to determine the specific height of the human object, and based on the determination result, locks whether the human object is a child.

2. A monitoring terminal for children riding alone in an elevator based on video analysis technology according to claim 1, characterized in that: The specific method of determining the moving object at the video analysis and processing end is: Based on the video segment associated with the start time period, starting from the initial moment of the video segment, confirm the initial frame, then confirm the subsequent frames that appear in the video segment in sequence, and generate a frame sequence according to the time sequence; Based on the determined frame sorting sequence, the characteristics of the pixels in each frame are confirmed, and the vertical gradient value T1 and the horizontal gradient value T2 of the corresponding pixel are preferentially determined. Confirm the feature Tz of the corresponding pixel point, and mark the pixel points with the same clustering feature in the adjacent frames as the same point position. The same clustering features include: the feature Tz confirmed by the corresponding pixel point is consistent, and the feature Tz associated with the surrounding pixel points with the same position feature of the corresponding pixel point is also consistent; Based on the same points marked in adjacent frames, the same points in different positions are marked as moving points: the corresponding frames are considered to be a set of planes, and the same points have position coordinates in these planes. When the position coordinates of this point change in the next set of frames, such same points are classified as moving points. Based on the moving points confirmed in succession in several frames in the frame sorting sequence, the objects covered by several adjacent moving points are marked as moving objects.

3. A monitoring terminal for children riding alone in an elevator based on video analysis technology according to claim 2, characterized in that: The method for determining the surrounding pixels of the corresponding pixel points having the same position feature is as follows: In the previous set of frames: take the corresponding pixel point as the starting point, confirm the location of the surrounding pixel points and use it as the end point, and confirm a set of vectors based on the starting point and the end point; In the next set of frames: the same method is used to confirm the vector, the surrounding pixels with the same vector features are marked as the surrounding pixels with the same position features, and the feature Tz consistency comparison is performed.

4. The terminal for monitoring a child riding an elevator alone based on video analysis technology according to claim 1, characterized in that: The specific method of determining the person object at the mobile object confirmation end is as follows: Confirm that the area ratio Zb of the corresponding moving object in each group of frames is Zb = the total area of ​​the moving object ÷ the total area of ​​the frames, mark the frame with the maximum area ratio Zb as the frame to be checked, mark the moving object in the frame to be checked as the object to be checked, determine the center point of the object to be checked, align the center point of the object to be checked with the center point of the human body contour template, the center point of the human body contour template is the preset point, and then the human body contour template is scaled proportionally according to the center point inside it, and stop when the overlap area between the object to be checked and the human body contour template reaches the maximum state. During the scaling process, the object to be checked is located in the human body contour template, and confirm that the overlap area is located in the coverage ratio of the human body contour template, the coverage ratio = the overlap area ÷ the total area of ​​the human body contour template. When the coverage ratio is ≥80%, the object to be checked is marked as a human object; Identify the number of human objects in this video segment. If the number = 1, execute the moving object confirmation end.

5. The terminal for monitoring a child riding an elevator alone based on video analysis technology according to claim 4 is characterized in that: When the number is ≥ 2, no processing is performed.

6. The terminal for monitoring a child riding an elevator alone based on video analysis technology according to claim 4, characterized in that: When the coverage ratio is less than 80%, no specific calibration of the person object is performed.

7. The terminal for monitoring a child riding an elevator alone based on video analysis technology according to claim 4, characterized in that: The specific method of the height determination end to determine whether the character object is a child is as follows: Based on the highest point preset in the human body contour template, the contour point closest to the highest point is identified from the human object, and the identified contour point is calibrated as the highest feature point. Then, based on the lowest points preset on the left and right sides of the human body contour template, the point closest to the lowest point is locked from the edge contour of the human object and calibrated as the lowest feature point. There are two groups of lowest feature points. The two groups of lowest feature points are connected to confirm a group of feature connection lines, and a vertical line A vertically downward from the highest feature point is constructed. The intersection of the vertical line A and the feature connection line is calibrated as the mapping base point A of the highest feature point. Based on the feature points of the real-time video recording end, confirm the vertical line B pointing vertically downward, and then determine the intersection of the vertical line B and the elevator edge line and mark it as the mapping base point B; Directly connect the mapping base point A and the mapping base point B, confirm the base point connection line, move the vertical line A according to the confirmed base point connection line, make the vertical line A coincide with the vertical line B, and confirm the virtual length C1 of the vertical line A and the virtual length C2 of the vertical line B, and use GD=C1÷C2 to confirm the height ratio value GD of the human object; The confirmed height proportion value GD is compared with the preset value Y1, where Y1 is the preset value. When GD≤Y1, the person object is calibrated as a child and a child signal is generated and transmitted to the monitoring and early warning center. When GD>Y1, no calibration is performed.

8. The terminal for monitoring a child riding an elevator alone based on video analysis technology according to claim 7, characterized in that: Also includes: The monitoring and early warning center issues real-time early warnings based on the child signals sent by the designated elevator, and conducts real-time video monitoring of the elevator while maintaining voice call functions.

Images