Method, device and electronic equipment for identifying escalator reverse movement

Abstract

The present invention provides a method, device and electronic device for identifying escalator reverse movement. The method uses speed decomposition to decompose the speed of a person's head into speeds along an escalator extension direction and speeds perpendicular to the escalator extension direction, thereby eliminating inaccurate speed recognition caused by pedestrians swaying left and right; uses a function to fit the speed of the pedestrian along the escalator extension direction at each position, thereby overcoming the problem of different pixel speeds of the pedestrian in an image due to different distances between the escalator position and a camera; finally, according to the comparison result of the fitting speed at each position and the pixel speed of the pedestrian along the escalator extension direction, it is judged whether the pedestrian is going in the reverse direction, thereby improving the accuracy of the judgment of the reverse movement.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an escalator reverse running recognition method, device and electronic equipment. Background Art

[0002] At present, escalators are used in shopping malls, subway stations, railway stations, airports and other places with dense traffic. While escalators bring great convenience to people, they can also cause loss of life and property due to improper use. For example, it is extremely dangerous for children to go against the flow on an escalator, which can easily lead to children falling and rolling, fingers being pinched and other serious consequences.

[0003] For the personal safety of pedestrians on escalators, the existing technology has developed some methods for real-time alarm of people's retrograde behavior on escalators, but these methods mainly realize the behavior recognition of people on escalators by adding various hardware such as sensors, which has high deployment cost and low recognition accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide an escalator reverse running identification method, device and electronic equipment, which can accurately calculate the speed of a person at various positions on the escalator along the escalator direction under different camera shooting angles, and accurately judge the escalator reverse running.

[0005] To achieve the above object, the present invention provides a method for identifying escalator reverse movement, comprising the following steps:

[0006] Step S1, acquiring an image to be detected, identifying the position of a head with an abnormal orientation in the image to be detected and the pixel speed of the head along the extension direction of the escalator;

[0007] Step S2, obtaining a historical data set, wherein the historical data set includes a plurality of first data obtained by identifying a plurality of historical images before the image to be detected, each of which includes a position of a head on the escalator in a historical image and a pixel speed of the head along the extension direction of the escalator when the head is at the position;

[0008] Step S3, obtaining a first function by fitting a plurality of first data, wherein the first function is used to represent the relationship between the fitting speed along the extension direction of the escalator in the image and the escalator position;

[0009] Step S4, obtaining a fitting speed of the abnormally oriented head along the extending direction of the escalator according to the position of the abnormally oriented head and the first function;

[0010] Step S5, determining the difference between the fitting speed and the pixel speed of the head with abnormal orientation along the extension direction of the escalator, and when the difference is less than a preset speed difference threshold, determining that the head is moving in the wrong direction.

[0011] Optionally, the step S1 specifically includes:

[0012] Perform target recognition on the image to be detected to obtain all the heads in the image to be detected;

[0013] Classify and detect all human heads in the image to be detected to obtain positions of human heads with abnormal orientations in the image to be detected;

[0014] Track the head with abnormal orientation, and calculate the pixel speed of the head with abnormal orientation according to the change of the head position before and after tracking;

[0015] The pixel speed of the abnormally oriented head is decomposed to obtain the pixel speed of the abnormally oriented head along the escalator extension direction and the pixel speed of the escalator extension direction perpendicular to the escalator extension direction.

[0016] Optionally, the step S2 specifically includes:

[0017] Extracting multiple historical images of a preset period of time before the image to be detected;

[0018] Identifying the positions and pixel speeds of all human heads in each historical image in the historical image set;

[0019] Decomposing the pixel speed of the head to obtain the pixel speed of the head along the escalator extension direction and the pixel speed of the head perpendicular to the escalator extension direction;

[0020] The position of a person's head on the escalator and the pixel speed of the person's head corresponding to the position along the extending direction of the escalator are used to form first data, so as to obtain a historical data set including a plurality of first data.

[0021] Optionally, the first function in step S3 is expressed as follows:

[0022] Vy = f(y);

[0023] Where f is a linear or quadratic function, y represents the position of the head, and Vy represents the fitting speed along the extension direction of the escalator when the head is at the y position.

[0024] Optionally, the formula for judging the reverse direction in step S5 is:

[0025] Vy-Vv <a*std

[0026] Among them, Vy is the fitting speed of the abnormal head along the escalator extension direction, Vv is the pixel speed of the abnormal head along the escalator extension direction, std is the standard deviation of the pixel speed along the escalator extension direction with the same position as the abnormal head in the historical data set, and a is an adjustable parameter.

[0027] Optionally, in step S1, the image to be detected comes from a first camera;

[0028] When the escalator is an escalator running towards the direction close to the first camera, the head with abnormal orientation is the head with the back facing;

[0029] When the escalator is an escalator running in a direction away from the first camera, the human head with an abnormal orientation is a frontal human head.

[0030] Optionally, the method further comprises:

[0031] Step S6, counting the length of time each person stays on the escalator except for the person who has been determined to be traveling in the wrong direction, and determining that the person is traveling in the wrong direction when the difference between the length of time a person stays and the preset regular passage time is greater than a preset time threshold.

[0032] Optionally, the regular passage time is determined by calculating the average time that each person in the historical image set stays in the escalator area.

[0033] The present invention also provides an escalator reverse identification device, comprising:

[0034] A detection unit, used to acquire an image to be detected, identify the position of a head with an abnormal orientation in the image to be detected and the pixel speed of the head along the extension direction of the escalator;

[0035] an acquisition unit, configured to acquire a historical data set, wherein the historical data set includes a plurality of first data obtained by identifying a plurality of historical images before the image to be detected, each of the first data including a position of a head on the escalator in a historical image and a pixel speed of the head along an extending direction of the escalator when the head is at the position;

[0036] A fitting unit, used for fitting a first function using a plurality of first data, wherein the first function is used for representing a relationship between a fitting speed along an extension direction of the escalator in the image and a position of the escalator;

[0037] a calculation unit, configured to obtain a fitting speed of the abnormally oriented human head along an extending direction of the escalator according to the position of the abnormally oriented human head and a first function;

[0038] The judgment unit is used to determine the difference between the fitting speed and the pixel speed of the head with abnormal direction along the extension direction of the escalator, and when the difference is less than a preset speed difference threshold, it is determined that the head is in the wrong direction.

[0039] The present invention also provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0040] The beneficial effects of the present invention are as follows: the present invention provides a method, device and electronic device for identifying escalator reverse movement. The method uses speed decomposition to decompose the speed of a person's head into speeds along the extension direction of the escalator and speeds perpendicular to the extension direction of the escalator, which can eliminate inaccurate speed recognition caused by pedestrians swaying left and right; a function is used to fit the speed of pedestrians along the extension direction of the escalator at each position, which can overcome the problem of different pixel speeds of pedestrians in the image due to different distances between the escalator position and the camera; finally, whether the pedestrian is going in the reverse direction is judged based on the comparison results of the fitting speed at each position and the pixel speed of the pedestrian along the extension direction of the escalator, which can improve the accuracy of the reverse movement judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings are only provided for reference and illustration and are not intended to limit the present invention.

[0042] In the accompanying drawings,

[0043] Figure 1 It is a flow chart of the method for identifying the reverse movement of an escalator of the present invention;

[0044] Figure 2 This is a flow chart of step S1 of the method for identifying escalator reverse movement of the present invention;

[0045] Figure 3 This is a flow chart of step S2 of the method for identifying escalator reverse movement of the present invention;

[0046] Figure 4 It is a flow chart of the direction classification in step S1 of the method for identifying the reverse movement of an escalator of the present invention;

[0047] Figure 5 Flow chart of step S5 of the escalator reverse identification method of the present invention

[0048] Figure 6 It is a schematic diagram of head recognition in the escalator reverse recognition method of the present invention;

[0049] Figure 7 It is a schematic diagram of speed decomposition in the method for identifying escalator reverse movement of the present invention;

[0050] Figure 8 It is a schematic diagram of direction recognition in the method for recognizing the reverse movement of an escalator of the present invention;

[0051] Fig. 9 It is a schematic diagram of an escalator reverse identification device of the present invention;

[0052] Fig.10 Schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0053] To further illustrate the technical means and effects of the present invention, the following is a detailed description in conjunction with the preferred embodiments of the present invention and the accompanying drawings.

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0055] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0056] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0057] See also Figure 1 The present invention provides a method for identifying an escalator running in reverse, comprising the following steps:

[0058] Step S1, obtaining an image to be detected, identifying the position of a human head with an abnormal orientation in the image to be detected and the pixel speed of the human head along the extension direction of the escalator.

[0059] Specifically, Figure 2 , Figure 5 , Figure 6 and Figure 7 As shown, the step S1 specifically includes:

[0060] Perform target recognition on the image to be detected to obtain all the heads in the image to be detected;

[0061] Classify and detect all human heads in the image to be detected to obtain positions of human heads with abnormal orientations in the image to be detected;

[0062] Track the head with abnormal orientation, and calculate the pixel speed of the head with abnormal orientation according to the change of the head position before and after tracking;

[0063] The pixel speed of the abnormally oriented head is decomposed to obtain the pixel speed of the abnormally oriented head along the escalator extension direction and the pixel speed of the escalator extension direction perpendicular to the escalator extension direction.

[0064] Among them, preferably, target recognition of the image to be detected is achieved through the yolov5s network model, classification and detection of all human heads in the image to be detected is completed through a pre-trained image classification network, and the method for tracking all human heads is achieved through the deepsort tracking method.

[0065] Furthermore, obtaining the image to be detected in step S1 specifically includes: reading the video stream, obtaining image frames from the video stream according to a preset frame extraction method, such as extracting one frame every 10 frames, and combining the extracted image frames to obtain a historical image set.

[0066] Furthermore, the video stream can be captured by a surveillance camera corresponding to the escalator to be monitored. The surveillance camera should be able to completely capture the area where the escalator is located. The running direction of the escalator can be toward the camera or away from the camera.

[0067] Specifically, the pixel speed of all human heads is calculated according to the change of the positions of the front and rear human heads, which specifically includes: determining the moving distance and the time consumed by the human heads according to the position coordinates of the front and rear human heads and the number of frames between the front and rear human heads, and then calculating the pixel speed of all human heads according to the distance and time.

[0068] Specifically, combined Figure 5 and Figure 8 In step S1, the image to be detected comes from the first camera;

[0069] When the escalator is an escalator running towards the direction close to the first camera, the head with abnormal orientation is the head with the back facing;

[0070] When the escalator is an escalator running in a direction away from the first camera, the human head with an abnormal orientation is a frontal human head.

[0071] The orientation of the human head can be identified through a classification network model.

[0072] like Figure 8 As shown, Figure 8 The middle escalator is going down (the escalator running towards the first camera). Figure 8 The back face in the figure is a head with an abnormal orientation and needs to be listed as a suspicious person for subsequent identification, while the front face has a normal orientation and does not need to be identified.

[0073] Step S2, obtaining a historical data set, wherein the historical data set includes a plurality of first data obtained by identifying a plurality of historical images before the image to be detected, each of which includes a position of a head on the escalator in a historical image and a corresponding pixel speed of the head along the extension direction of the escalator when the head is at that position.

[0074] Specifically, Figure 3 As shown, the step S2 specifically includes:

[0075] Extracting multiple historical images of a preset period of time before the image to be detected;

[0076] Identifying the positions and pixel speeds of all human heads in each historical image in the historical image set;

[0077] Decomposing the pixel speed of the head to obtain the pixel speed of the head along the escalator extension direction and the pixel speed of the head perpendicular to the escalator extension direction;

[0078] The position of a person's head on the escalator and the pixel speed of the person's head corresponding to the position along the extending direction of the escalator are used to form first data, so as to obtain a historical data set including a plurality of first data.

[0079] Specifically, in step S3, the data points are represented as follows: (y1, Vy1), (y2, Vy2), (y3, Vy3), (yn, Vyn), wherein y1, y2, y3 and yn represent the position of the head on the escalator, and Vy1, Vy2, Vy3 and Vyn represent the pixel speed of the head at this position along the extension direction of the escalator.

[0080] It should be noted that, in step S1 and step S2, the pixel speed of the head at each position is decomposed into the pixel speed Vy of the head along the extension direction of the escalator and the pixel speed Vy of the head perpendicular to the extension direction of the escalator. Each position includes the pixel speeds of multiple heads passing through over a period of time, and thus the pixel speeds Vy of multiple heads along the extension direction of the escalator at different time points can be obtained for each position.

[0081] Among them, the use of speed decomposition can eliminate the situation where the speed fitting is inaccurate due to the pedestrian's swaying left and right.

[0082] Step S3: obtaining a first function by fitting a plurality of first data, wherein the first function is used to represent the relationship between the fitting speed along the extension direction of the escalator in the image and the escalator position.

[0083] Specifically, the first function in step S3 is expressed as follows:

[0084] Vy = f(y);

[0085] Where f is a linear or quadratic function, y represents the position of the head, and Vy represents the fitting speed along the extension direction of the escalator when the head is at the y position.

[0086] Step S4: obtaining a fitting speed of the abnormally oriented human head along the extending direction of the escalator according to the position of the abnormally oriented human head and the first function.

[0087] Specifically, combined Figure 8 , Figure 7 The back face in the figure is the head facing abnormally. Figure 8 A check frame for the back face is drawn, and the position y of the back face is determined according to the check frame, and is substituted into the first function to obtain a fitting speed Vy of the back face along the extension direction of the escalator.

[0088] Among them, the fitting speed is obtained by fitting the function, which can overcome the problem of different pixel speeds of pedestrians in the image due to the different distances between the escalator and the camera.

[0089] Step S5, determining the difference between the fitting speed and the pixel speed of the head with abnormal orientation along the extension direction of the escalator, and when the difference is less than a preset speed difference threshold, determining that the head is moving in the wrong direction.

[0090] Specifically, Figure 4 As shown, the step S5 specifically includes:

[0091] Determine the standard deviation of the pixel speed along the extension direction of the escalator in the historical data set that is the same as the position of the head of the person with the abnormal orientation, and determine the speed difference threshold value according to the standard deviation;

[0092] Calculate the difference between the fitting speed and the pixel speed of the head facing the abnormal direction along the escalator extension direction;

[0093] The difference is compared with a speed difference threshold, and it is determined whether the head of the person with abnormal orientation is traveling in the reverse direction according to the comparison result.

[0094] Specifically, the formula for judging whether the vehicle is traveling in the reverse direction in step S5 is:

[0095] Vy-Vv <a*std

[0096] Among them, Vy is the fitting speed of the abnormal head along the escalator extension direction, Vv is the pixel speed of the abnormal head along the escalator extension direction, std is the standard deviation of the pixel speed along the escalator extension direction of the same position as the abnormal head in the historical image set, and a is an adjustable parameter. Generally speaking, the adjustable parameter of the abnormal head is smaller than that of the normal head.

[0097] Furthermore, in order to avoid missed identification due to someone deliberately standing in the opposite direction and walking in the wrong direction, some embodiments of the present invention also include the following steps: Step S8, counting the length of time each person stays on the escalator except for the person who has been determined to be walking in the wrong direction, and when the difference between the length of time a person stays and the preset regular passage time is greater than a preset time threshold, the person is determined to be walking in the wrong direction.

[0098] The conventional passage time is determined by calculating the average time that each person in the historical image set stays in the escalator area.

[0099] The length of time a person stays in the escalator area can be used to assist in determining whether there is retrograde behavior, which can further increase the accuracy of detection and identification.

[0100] See also Fig. 9 The present invention also provides an escalator reverse identification device, comprising:

[0101] The detection unit 10 is used to obtain the image to be detected, identify the position of the head of the person facing abnormally in the image to be detected and the pixel speed of the head along the extension direction of the escalator;

[0102] An acquisition unit 20 is used to acquire a historical data set, wherein the historical data set includes a plurality of first data obtained by identifying a plurality of historical images before the image to be detected, each of which includes a position of a head on the escalator in a historical image and a pixel speed of the head along the extension direction of the escalator when the head is at the position;

[0103] A fitting unit 30, configured to obtain a first function by fitting the plurality of first data, wherein the first function is used to represent a relationship between a fitting speed along an extension direction of the escalator in the image and a position of the escalator;

[0104] A calculation unit 40, configured to obtain a fitting speed of the abnormally oriented human head along an extending direction of the escalator according to the position of the abnormally oriented human head and the first function;

[0105] The judgment unit 50 is used to determine the difference between the fitting speed and the pixel speed of the head with abnormal direction along the extension direction of the escalator, and when the difference is less than a preset speed difference threshold, it is determined that the head is in the wrong direction.

[0106] See also Fig.10 The present invention also provides an electronic device, comprising: a memory 100 and a processor 200, wherein the memory 100 stores a computer program, and when the computer program is executed by the processor 200, the processor 200 executes the steps of the above method.

[0107] In summary, the present invention provides a method, device and electronic device for identifying escalator reverse movement. The method uses speed decomposition to decompose the speed of a person's head into speeds along the extension direction of the escalator and speeds perpendicular to the extension direction of the escalator, which can eliminate inaccurate speed recognition caused by pedestrians swaying left and right; uses a function to fit the speed of pedestrians along the extension direction of the escalator at each position, which can overcome the problem of different pixel speeds of pedestrians in the image due to different distances between the escalator position and the camera; finally, based on the comparison results of the fitting speed at each position and the pixel speed of the pedestrian along the extension direction of the escalator, it is judged whether it is reverse movement, which can improve the accuracy of reverse movement judgment.

[0108] As described above, for ordinary technicians in this field, various other corresponding changes and modifications can be made based on the technical solutions and technical concepts of the present invention, and all these changes and modifications should fall within the protection scope of the claims of the present invention.

Claims

1. A method for identifying escalator reverse movement, characterized in that: The steps include: Step S1, acquiring an image to be detected, identifying the position of a head with an abnormal orientation in the image to be detected and the pixel speed of the head along the extension direction of the escalator; Step S2, obtaining a historical data set, wherein the historical data set includes a plurality of first data obtained by identifying a plurality of historical images before the image to be detected, each of which includes a position of a head on the escalator in a historical image and a pixel speed of the head along the extension direction of the escalator when the head is at the position; Step S3, obtaining a first function by fitting the plurality of first data, wherein the first function is used to represent the relationship between the fitting speed along the extension direction of the escalator in the image and the escalator position; Step S4, obtaining a fitting speed of the abnormally oriented head along the extending direction of the escalator according to the position of the abnormally oriented head and the first function; Step S5, determining the difference between the fitting speed and the pixel speed of the head with abnormal orientation along the extension direction of the escalator, and when the difference is less than a preset speed difference threshold, determining that the head is moving in the wrong direction.

2. The method for identifying escalator reverse movement according to claim 1, characterized in that: The step S1 specifically includes: Perform target recognition on the image to be detected to obtain all the heads in the image to be detected; Classify and detect all human heads in the image to be detected to obtain positions of human heads with abnormal orientations in the image to be detected; Track the head with abnormal orientation, and calculate the pixel speed of the head with abnormal orientation according to the change of the head position before and after tracking; The pixel speed of the abnormally oriented head is decomposed to obtain the pixel speed of the abnormally oriented head along the escalator extension direction and the pixel speed of the escalator extension direction perpendicular to the escalator extension direction.

3. The method for identifying escalator reverse movement according to claim 1, characterized in that: The step S2 specifically includes: Extracting multiple historical images of a preset period of time before the image to be detected; Identifying the positions and pixel speeds of all human heads in each historical image in the historical image set; Decomposing the pixel speed of the head to obtain the pixel speed of the head along the escalator extension direction and the pixel speed of the head perpendicular to the escalator extension direction; The position of a person's head on the escalator and the pixel speed of the person's head corresponding to the position along the extending direction of the escalator are used to form first data, so as to obtain a historical data set including a plurality of first data.

4. The method for identifying escalator reverse movement according to claim 1, characterized in that: The first function in step S3 is expressed as follows: Vy = f(y); Where f is a linear or quadratic function, y represents the position of the head, and Vy represents the fitting speed along the extension direction of the escalator when the head is at the y position.

5. The method for identifying escalator reverse movement according to claim 1, characterized in that: The formula for judging the reverse direction in step S5 is: Vy-Vv <a*std Among them, Vy _ Vv is the fitting speed of the head towards the abnormality along the escalator extension direction, _ is the pixel speed of the abnormal head along the escalator extension direction, std is the standard deviation of the pixel speed along the escalator extension direction in the historical data set with the same position as the abnormal head, and a is an adjustable parameter.

6. The method for identifying escalator reverse movement according to claim 1, characterized in that: In the step S1, the image to be detected comes from the first camera; When the escalator is an escalator running towards the direction close to the first camera, the head with abnormal orientation is the head with the back facing; When the escalator is an escalator running in a direction away from the first camera, the human head with an abnormal orientation is a frontal human head.

7. The method for identifying escalator reverse movement according to claim 1, characterized in that: Also includes: Step S6, counting the length of time each person stays on the escalator except for the person who has been determined to be traveling in the wrong direction, and determining that the person is traveling in the wrong direction when the difference between the length of time a person stays and the preset regular passage time is greater than a preset time threshold.

8. The method for identifying escalator reverse movement according to claim 7, characterized in that: The conventional passage time is determined by calculating the average time that each person in the historical image set stays in the escalator area.

9. An escalator reverse identification device, characterized in that: include: A detection unit, used to acquire an image to be detected, identify the position of a head with an abnormal orientation in the image to be detected and the pixel speed of the head along the extension direction of the escalator; an acquisition unit, configured to acquire a historical data set, wherein the historical data set includes a plurality of first data obtained by identifying a plurality of historical images before the image to be detected, each of the first data including a position of a head on the escalator in a historical image and a pixel speed of the head along an extending direction of the escalator when the head is at the position; A fitting unit, used for fitting a first function using a plurality of first data, wherein the first function is used for representing a relationship between a fitting speed along an extension direction of the escalator in the image and a position of the escalator; a calculation unit, configured to obtain a fitting speed of the abnormally oriented human head along an extending direction of the escalator according to the position of the abnormally oriented human head and a first function; The judgment unit is used to determine the difference between the fitting speed and the pixel speed of the head with abnormal direction along the extension direction of the escalator, and when the difference is less than a preset speed difference threshold, it is determined that the head is in the wrong direction.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.

Images