Locomotive crew normative behavior analysis method, device, equipment and storage medium

Abstract

The present application relates to a method, device, equipment and storage medium for analyzing the standardized behavior of locomotive crew members, the method comprising obtaining monitoring videos of locomotive crew members during locomotive operation, and obtaining the target time when the locomotive crew members need to have standardized behavior from the train operation monitoring information; determining the monitoring period corresponding to the target time; intercepting the target monitoring video corresponding to the monitoring period from the monitoring video; detecting the key areas in each video frame image in the target monitoring video; and determining the standardized behavior of the locomotive crew members based on the key areas. The scheme of the present application is adopted to realize automatic analysis of the standardized behaviors of the locomotive crew members such as standing up behavior, hand gesture behavior and rear lookout behavior, reduce the workload of manual video review, and improve work efficiency.

Description

Technical Field

[0001] The present application relates to the field of rail transit and locomotive driving, and in particular to a method, device, equipment and storage medium for analyzing the standardized behavior of locomotive crew members. Background Art

[0002] As the driver of the locomotive, the driving behavior of the railway locomotive crew directly determines whether the locomotive can operate safely. Therefore, it is of great significance to analyze the operating behavior of the locomotive crew to achieve standardized operation and safe driving.

[0003] In the related technology, the monitoring video of the locomotive crew is manually reviewed to judge whether the locomotive crew's operating behavior is standardized. The problem is that the manual review is heavy and inefficient, and it cannot fully cover the video and personnel review and analysis, and there are problems of missed inspections. Summary of the invention

[0004] The present application provides a method, device, equipment and storage medium for analyzing the standardized behavior of locomotive crew members, which is used to solve the problem of large workload of manual review in related technologies, inability to achieve full coverage of videos and personnel, and easy omissions in inspections.

[0005] A first aspect is to provide a method for analyzing the standardized behavior of a locomotive crew member, comprising:

[0006] Obtaining monitoring video of the locomotive crew during the operation of the locomotive, and obtaining the target time when the locomotive crew needs to perform specified standardized behavior from the train operation monitoring information;

[0007] Determining a monitoring period corresponding to the target time;

[0008] Intercepting a target surveillance video corresponding to the surveillance period from the surveillance video;

[0009] Detecting key areas in each video frame image in the target monitoring video, wherein the key areas include a human body key area and a cab key area;

[0010] The standard behaviors of the locomotive crew are determined based on the key areas. The standard behaviors are prescribed behaviors that the locomotive crew must complete during the process of leaving the warehouse, departing, and running. The standard behaviors include hand gestures, rear lookout, and standing up.

[0011] Optionally, intercepting a target surveillance video corresponding to the surveillance period from the surveillance video includes:

[0012] Based on OCR recognition, obtaining the acquisition time of each video frame image in the surveillance video, the OCR recognition content including but not limited to time and kilometer mark;

[0013] According to the acquisition time, all video frame images within the monitoring period are determined to be the target monitoring video.

[0014] Optionally, determining the standard behavior of the locomotive crew according to the key area includes:

[0015] For any video frame image in the target surveillance video, an arm region and a hand region in the video frame image are obtained, wherein the arm region and the hand region belong to the key region of the human body;

[0016] Determine whether the aspect ratio of the arm region is greater than an aspect ratio threshold, and whether the ordinate of the center point of the hand region is not greater than the ordinate of the center point of the arm region;

[0017] If so, determine that the standard behavior of the locomotive crew is the hand gesture behavior.

[0018] Optionally, before determining that the behavior of the locomotive crew is the hand gesture behavior, the method further includes:

[0019] Acquire a locomotive crew region in the video frame image, where the locomotive crew region belongs to the key human body region;

[0020] respectively calculating a first degree of overlap between the arm region and the hand region, and a second degree of overlap between the arm region and the locomotive crew region;

[0021] It is determined that the first overlap degree and the second overlap degree are respectively greater than an overlap degree threshold.

[0022] Optionally, determining the standard behavior of the locomotive crew according to the key area includes:

[0023] For any video frame image in the target surveillance video, obtaining the back area, side window area and door area of ​​the locomotive crew in the video frame image;

[0024] Determine whether the relative distance between the back area of ​​the locomotive crew and the side window area or the door area is not greater than a distance threshold, the back area of ​​the locomotive crew belongs to the key area of ​​the human body, and the side window area and the door area belong to the key area of ​​the cab;

[0025] If so, determine that the standard behavior of the locomotive crew is the rear lookout behavior.

[0026] Optionally, determining the standard behavior of the locomotive crew according to the key area includes:

[0027] For any video frame image in the target monitoring video, obtaining the height of the locomotive crew member in the video frame image;

[0028] Determining whether the height of the locomotive crew member is greater than a height threshold;

[0029] If so, determine that the standard behavior of the locomotive crew is the standing up behavior.

[0030] Optionally, determining whether the height of the locomotive crew member is greater than a height threshold includes:

[0031] Acquire the head area of ​​the locomotive crew in the video frame image;

[0032] Determining the area of ​​the head region and the area of ​​the video frame image;

[0033] Calculating an area ratio of the head region relative to the video frame image;

[0034] determining the interval to which the area ratio belongs;

[0035] Using the height threshold corresponding to the interval as the height threshold used by the video frame image;

[0036] It is determined that the height of the locomotive crew member is greater than the height threshold.

[0037] The second aspect is to provide a standardized behavior analysis device for a locomotive crew, comprising:

[0038] An acquisition unit, used to acquire monitoring video of the locomotive crew during the operation of the locomotive, and to acquire a target time when the locomotive crew needs to have a standardized behavior from the train operation monitoring information;

[0039] A first determining unit, configured to determine a monitoring period corresponding to the target time;

[0040] An interception unit, used to intercept a target surveillance video corresponding to the surveillance period from the surveillance video;

[0041] A detection unit, used to detect key areas in each video frame image in the target monitoring video, wherein the key areas include a human body key area and a cab key area;

[0042] The second determination unit is used to determine the standard behavior of the locomotive crew according to the key area. The standard behavior is the prescribed behavior that the locomotive crew must complete during the process of shunting, departure, operation, etc. The standard behavior includes hand gestures, rear lookout behavior or standing up behavior.

[0043] A third aspect provides an electronic device, comprising: a processor, a memory and a communication bus, wherein the processor and the memory communicate with each other through the communication bus;

[0044] The memory is used to store computer programs;

[0045] The processor is used to execute the program stored in the memory to implement the method for analyzing the standardized behavior of locomotive crew members described in the first aspect.

[0046] A fourth aspect provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method for analyzing the standardized behavior of locomotive crew members described in the first aspect.

[0047] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the technical solution provided by the embodiment of the present application obtains the monitoring video of the locomotive crew during the operation of the locomotive, and obtains the target time when the locomotive crew needs to have standardized behavior from the train operation monitoring information; determines the monitoring period corresponding to the target time; intercepts the target monitoring video corresponding to the monitoring period from the monitoring video; detects the key areas in each video frame image in the target monitoring video, and the key areas include the key areas of the human body and the key areas of the cab; determines the standardized behavior of the locomotive crew according to the key areas. The solution of the present application is adopted to realize the automatic analysis of the standardized behaviors of the locomotive crew such as standing up behavior, hand gesture behavior and rear lookout behavior, reduce the workload of manual video review, and improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 A schematic diagram of a flow chart of a method for analyzing the standardized behavior of a locomotive crew member in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a flow chart of a method for analyzing hand gesture behavior of a locomotive crew member in an embodiment of the present application;

[0052] Figure 3 It is another flow chart of the method for analyzing the hand gesture behavior of a locomotive crew member in the embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of the effect of calculating the degree of overlap shown in an embodiment of the present application;

[0054] Figure 5 A flowchart of a method for analyzing the rear lookout behavior of a locomotive crew member for implementing this application;

[0055] Figure 6 It is another effect schematic diagram of calculating the relative distance between the back area of ​​a train crew member and the side window area, or the back area of ​​a locomotive crew member and the door area shown in the embodiment of the present application;

[0056] Figure 7 It is another effect schematic diagram of calculating the relative distance between the back area of ​​a train crew member and the side window area, or the back area of ​​a locomotive crew member and the door area shown in the embodiment of the present application;

[0057] Figure 8 It is another effect schematic diagram of calculating the relative distance between the back area of ​​a train crew member and the side window area, or the back area of ​​a locomotive crew member and the door area shown in the embodiment of the present application;

[0058] Fig. 9 A schematic diagram of a flow chart of a method for analyzing the standing-up behavior of a locomotive crew member in an embodiment of the present application;

[0059] Fig.10 A schematic diagram showing an effect of calculating the height of a train attendant shown in an embodiment of the present application;

[0060] Fig.11 It is another flow chart of the method for analyzing the standing-up behavior of a locomotive crew member in the embodiment of the present application;

[0061] Fig.12 This is a schematic diagram of the structure of a device for analyzing the standardized behavior of a locomotive crew member in an embodiment of the present application;

[0062] Fig.13 Schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0064] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] The embodiment of the present application provides a method for analyzing the normative behavior of a locomotive crew member, which can be applied to electronic equipment;

[0066] The electronic device may be any electronic device inside the locomotive, or any electronic device listed outside the locomotive, which is not specifically limited in this embodiment;

[0067] The electronic devices may include smart phones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, video matrices, monitoring platforms, mobile Internet devices (MID, Mobile Internet Devices) or wearable devices, etc. The above are only examples and not exhaustive, including but not limited to the above devices.

[0068] like Figure 1 As shown, the method for analyzing the standard behavior of a locomotive crew member may include the following steps:

[0069] Step 101, obtaining monitoring video of the locomotive crew during the locomotive operation, and obtaining the target time when the locomotive crew needs to have standardized behavior from the train operation monitoring information;

[0070] In this embodiment, the existing locomotive cab video surveillance can be used to collect surveillance video, the video surveillance communicates with the computer device, and the video surveillance transmits the corresponding surveillance video to the computer device through a communication bus or a network.

[0071] Among them, the video surveillance can be located at the right rear of the locomotive crew.

[0072] In this embodiment, the locomotive is provided with a train operation monitoring device (LKJ) for recording the locomotive operation status, signal equipment status and locomotive crew operating status, so the train operation monitoring information can be obtained from the train operation monitoring device.

[0073] Step 102: determine a monitoring period corresponding to the target time;

[0074] In this embodiment, the floating time can be preset, and the monitoring period can be obtained based on the floating time and the target time. For example, the target time obtained from the train operation monitoring information is set to 2021 / 5 / 27 10:00:00, and the floating time is 5s, then the determined monitoring period can be 2021 / 5 / 27 9:59:55-2021 / 5 / 27 10:00:05.

[0075] In this embodiment, the preset floating time corresponding to different standard behaviors may be different, that is, a preset floating time may be set for each type of standard behavior; or all standard behaviors may be set to correspond to the same preset floating time, which is not specifically limited in this embodiment.

[0076] Step 103: intercepting a target surveillance video corresponding to the surveillance period from the surveillance video;

[0077] In this embodiment, the target surveillance video can be intercepted from the surveillance video based on the acquisition time of each video frame image in the surveillance video. Specifically, the video composed of the video frame images whose acquisition time is in the surveillance period is determined as the target surveillance video.

[0078] Among them, the acquisition time of each video frame image can be obtained based on OCR (Optical Character Recognition), and the recognition content includes but is not limited to the current time and kilometer mark. Based on OCR recognition of information such as time and kilometer mark in the monitoring video, any frame of the video frame is associated with the LKJ information. The monitoring video and LKJ are made consistent in time, and then the target time when the locomotive crew needs to have standardized behavior is determined based on the LKJ information, and the corresponding target monitoring video is intercepted.

[0079] It should be understood that the monitoring period in this embodiment is actually a closed interval time period, so the video frame images corresponding to the two endpoint values ​​of the monitoring period should also belong to the video frame images in the target monitoring video.

[0080] Step 104: Detect key areas in each video frame image in the target monitoring video, where the key areas include a human body key area and a cab key area;

[0081] Step 105: Determine the standard behavior of the locomotive crew according to the key areas. The standard behavior is the prescribed behavior that the locomotive crew must complete during the crew operation. The standard behavior includes hand gestures, rear lookout, and standing up.

[0082] In the application, crew operations include processes such as departure from the warehouse, departure, and operation.

[0083] Optionally, key areas of the human body include but are not limited to the locomotive crew area, the locomotive crew back area, the face area, the face area of ​​people wearing masks, the arm area, the head area and the hand area; key areas of the cab include but are not limited to the seat area, the side window area, the door area (divided into open and closed states), and the operating console area.

[0084] The technical solution provided by the embodiment of the present application obtains the monitoring video of the locomotive crew during the operation of the locomotive, and obtains the target time when the locomotive crew needs to have standardized behavior from the train operation monitoring information; determines the monitoring period corresponding to the target time; intercepts the target monitoring video corresponding to the monitoring period from the monitoring video; detects the key areas in each video frame image in the target monitoring video, and the key areas include the key areas of the human body and the key areas of the cab; determines the standardized behavior of the locomotive crew according to the key areas. The solution of the present application realizes the automatic analysis of the standardized behaviors of the locomotive crew such as standing up behavior, hand gesture behavior, and rear lookout behavior, reduces the workload of manual video review, and improves work efficiency.

[0085] The following is a detailed description of the analysis process for each type of normative behavior:

[0086] Regarding hand gestures, hand gestures refer to the posture of the locomotive crew with arms extended and hands raised when the locomotive reaches a specific position. When the locomotive crew performs such standard behaviors, the locomotive crew's arms and hands have specific postures, so it can be determined whether the locomotive crew has hand gestures based on the postures of the arms and hands.

[0087] In the embodiment of the present application, the coordinate system of the video frame image is defined as follows:

[0088] From the perspective of image processing, the origin of the coordinate system is located at the upper left corner of the video frame image. The X-axis is established horizontally to the right from the origin, and the Y-axis is established vertically downward from the origin.

[0089] From the perspective of image processing, the width of the arm (hand) region is determined according to the horizontal direction of the video frame image, and the height of the arm (hand) region is determined according to the vertical direction of the video frame image.

[0090] Specifically, Figure 2 As shown, when determining whether the locomotive crew has hand gesture behavior, step 105 may include the following steps:

[0091] Step 201: for any video frame image in the target monitoring video, obtain the arm area and the hand area in the video frame image, where the arm area and the hand area belong to the key areas of the human body;

[0092] Step 202, determining whether the aspect ratio of the arm region is greater than the aspect ratio threshold, and whether the ordinate of the center point of the hand region is not greater than the ordinate of the center point of the arm region, if so, executing step 203; otherwise, returning to step 201;

[0093] Step 203: Determine that the standard behavior of the locomotive crew is the hand gesture behavior.

[0094] When the aspect ratio of the arm area is greater than the aspect ratio threshold and the ordinate of the center point of the hand area is not greater than the ordinate of the center point of the arm area, it is determined that the locomotive attendant has a posture with his arms extended and his hands raised, that is, the locomotive attendant has hand gesture behavior.

[0095] Among them, when the aspect ratio of the arm area is not greater than the aspect ratio threshold, or the vertical coordinate of the center point of the hand area is greater than the vertical coordinate of the center point of the arm area, it is determined that the locomotive crew did not show any hand gesture behavior in the video frame image.

[0096] Considering that there may be other staff members in the locomotive cab in addition to the locomotive crew, after obtaining the arm region and the hand region, it is possible to further determine whether the arm region and the hand region belong to the corresponding locomotive crew region.

[0097] Specifically, Figure 3 As shown, before step 203, the process may include the following steps:

[0098] Step 301, obtaining a locomotive crew region in a video frame image, where the locomotive crew region belongs to a key region of the human body;

[0099] Step 302, respectively calculate the first degree of overlap between the arm region and the hand region, and the second degree of overlap between the arm region and the locomotive crew region. The arm region includes shoulders, arms, elbows, forearms and hands. The locomotive crew region includes head, shoulders, abdomen, shoulders, arms, elbows, forearms and hands. When there are multiple arm regions in the monitoring video, the arm region of the locomotive crew is determined by the degree of overlap. When a first degree of overlap is greater than a threshold and a maximum value is obtained, the arm region corresponding to the first degree of overlap is determined to be the arm region of the locomotive crew; similarly, when a second degree of overlap is greater than a threshold and a maximum value is obtained, the hand region corresponding to the second degree of overlap is determined to be the hand region of the locomotive crew. The degree of overlap is calculated as the area ratio of the intersection of two regions to the union of the two regions, such as Figure 4 shown.

[0100] exist Figure 4 middle,

[0101] Step 303: Determine whether the first overlap degree and the second overlap degree are respectively greater than an overlap degree threshold.

[0102] In this embodiment, the first overlap degree is the ratio of the overlapping area of ​​the arm area and the hand area to the area of ​​the arm area and the hand area; the second overlap degree is the ratio of the overlapping area of ​​the arm area and the locomotive crew area to the area of ​​the arm area and the locomotive crew area.

[0103] In this embodiment, the overlap thresholds adapted by the first overlap and the second overlap may be the same or different, and this embodiment does not specifically limit this.

[0104] If there are two or more second overlaps in the locomotive crew area, and both satisfy a second overlap threshold, it is determined that the arm area corresponding to the maximum second overlap belongs to the corresponding locomotive crew area;

[0105] If two or more first overlaps occur in the arm region and both satisfy a first overlap threshold, it is determined that the hand region corresponding to the maximum first overlap belongs to the corresponding arm region.

[0106] Regarding the rear lookout behavior, the rear lookout behavior refers to that when the monitoring is opposite the window or door, that is, on the other side of the cab, the back area of ​​the locomotive crew is located at the side window or door of the cab. Therefore, when there is a video frame image in the target monitoring video in which the relative distance between the back area and the side window area or door area of ​​the cab is not greater than the distance threshold, it is determined that the locomotive crew has a rear lookout behavior.

[0107] Specifically, Figure 5 As shown, step 105 may include the following steps:

[0108] Step 501: for any video frame image in the target monitoring video, obtain the back area, side window area and door area of ​​the locomotive crew in the video frame image;

[0109] Step 502: determine whether the relative distance between the back area of ​​the locomotive crew and the side window area or the door area is not greater than the distance threshold. The back area of ​​the locomotive crew belongs to the key area of ​​the human body, and the side window area and the door area belong to the key area of ​​the cab. If so, execute step 503; otherwise, return to step 501;

[0110] Step 503: Determine that the standard behavior of the locomotive crew is the rear lookout behavior.

[0111] Among them, when the relative distance between the back area of ​​the locomotive crew and any of the side window area or the door area is not greater than the distance threshold, it is determined that the locomotive crew is looking out of the window or door. Regarding the setting of the distance threshold, the distance threshold can use a fixed value; considering that different surveillance video frame sizes may not be fixed values; the distance threshold can be defined according to a ratio or by calculating the distance between the seat and the side window or door.

[0112] In this embodiment, the following three methods are provided for calculating the relative distance between the back area of ​​the train crew and the side window area, or between the back area of ​​the locomotive crew and the door area:

[0113] First, please see Figure 6 , determine the center point of the back area of ​​the locomotive crew (hereinafter referred to as the first center point), determine the center point of the side window area (hereinafter referred to as the second center point), and determine the center point of the door area (hereinafter referred to as the third center point);

[0114] Determine the distance between the first center point and the second center point, and use the distance as the relative distance between the back area of ​​the locomotive crew and the side window area; or determine the distance between the first center point and the third center point, and use the distance as the relative distance between the back area of ​​the locomotive crew and the door area.

[0115] Second, please see Figure 7 , calculate the overlap between the back area of ​​the train crew and the side window area, and use the overlap as the relative distance between the back area of ​​the locomotive crew and the side window area; or calculate the overlap between the back area of ​​the locomotive crew and the door area, and use the overlap as the relative distance between the back area of ​​the locomotive crew and the door area.

[0116] The calculation of the overlap degree in this manner can refer to the aforementioned calculation of the first overlap degree and the calculation of the second overlap degree, which will not be described in detail here.

[0117] Third, please see Figure 8 , project the back area and the side window area of ​​the locomotive crew respectively in the horizontal direction to obtain two projection line segments, calculate the overlapping length of the two projection line segments, and use the distance as the relative distance between the back area and the side window area of ​​the locomotive crew; or, project the back area and the door area of ​​the locomotive crew respectively in the horizontal direction to obtain two projection line segments, calculate the overlapping length of the two projection line segments, and use the distance as the relative distance between the back area and the door area of ​​the locomotive crew.

[0118] It should be understood that any one of the above three methods can be selected to obtain the relative distance between the back area of ​​the locomotive crew and the side window area or the door area as needed, and this embodiment does not specifically limit this.

[0119] Regarding standing up behavior, the standing up behavior judgment standard is that the height of the locomotive crew in the video frame image is greater than the height threshold. Therefore, when there is a video frame image with a height greater than the height threshold in the target monitoring video, it is determined that the locomotive crew has stood up.

[0120] Specifically, Fig. 9As shown, step 105 may include the following steps:

[0121] Step 901: for any video frame image in the target monitoring video, obtain the height of the locomotive crew in the video frame image;

[0122] Step 902, determine whether the height of the locomotive crew is greater than the height threshold, if so, execute step 903;

[0123] Step 903: Determine that the standard behavior of the locomotive crew is standing up.

[0124] In this embodiment, Fig.10 As shown, when obtaining the height of the locomotive crew, determine the rectangular area where the locomotive crew is located, refer to the coordinate system of the video frame image described in the above content, obtain the vertical coordinate of the vertex in the upper left corner of the locomotive crew area and the height of the video frame image, calculate the difference between the height of the video frame image and the vertical coordinate of the vertex in the upper left corner, and use the difference as the height of the locomotive crew.

[0125] In this embodiment, the vertical coordinate of the upper left corner vertex of the locomotive crew area may be used, or the vertical coordinate of the upper right corner vertex may be used, or the vertical coordinate of the upper left and right corner vertices of the head area may be used, and this embodiment does not make any specific limitation on this.

[0126] When the video surveillance in the cab shoots a locomotive crew member at a long distance, the complete upper body of the locomotive crew member can be shot, and the head area is small; when shooting a locomotive crew member at a short distance, only the head of the locomotive crew member may be monitored, and the head area is large. Therefore, the distance relationship of the locomotive crew member relative to the video surveillance can be judged according to the area of ​​the locomotive crew member's head area in the current video frame image, and different height thresholds can be set according to the distance relationship, so as to use the height threshold to judge whether there is a locomotive crew member standing up in the current video frame image.

[0127] Specifically, Fig.11 As shown, step 902 may include the following steps:

[0128] Step 1101: Acquire the head region of the locomotive crew in the video frame image;

[0129] Step 1102: determine the area of ​​the head region and the area of ​​the video frame image;

[0130] Step 1103, calculating the area ratio of the head region to the video frame image;

[0131] Step 1104, determine the interval to which the area ratio belongs; determine the reference height based on the door, the side window and the seat. Set a piecewise function based on the reference height, and determine the segmented interval to which the area ratio belongs;

[0132] Step 1105: using the height threshold corresponding to the interval as the height threshold used by the video frame image;

[0133] Step 1106: Determine whether the height of the locomotive crew member is greater than the height threshold.

[0134] Among them, different intervals reflect the relative distance between the locomotive crew and the video surveillance. Different height thresholds are used at different relative distances to determine whether the locomotive crew has completed the standing behavior.

[0135] In the present application, the above-mentioned examples can be executed sequentially, but the order of their execution is not limited, and they can also be executed independently.

[0136] Based on the same concept, a device for analyzing the normative behavior of a locomotive crew is provided in the embodiment of the present application. The specific implementation of the device can be found in the description of the method embodiment part, and the repeated parts will not be repeated. Fig.12 As shown, the device mainly includes:

[0137] The acquisition unit 1201 is used to acquire the monitoring video of the locomotive crew during the locomotive operation, and to acquire the target time when the locomotive crew needs to have standardized behavior from the train operation monitoring information;

[0138] A first determining unit 1202, configured to determine a monitoring period corresponding to a target time;

[0139] The interception unit 1203 is used to intercept the target surveillance video corresponding to the surveillance period from the surveillance video;

[0140] The detection unit 1204 is used to detect the key area in each video frame image in the target monitoring video, and the key area includes the key area of ​​the human body and the key area of ​​the cab;

[0141] The second determining unit 1205 is used to determine the standard behavior of the locomotive crew according to the key area. The standard behavior is the prescribed behavior that the locomotive crew must complete during the crew operation, and the standard behavior includes hand gesture behavior, rear lookout behavior or standing behavior.

[0142] Optionally, the acquiring unit 1201 is specifically configured to:

[0143] Based on OCR recognition, the acquisition time of each video frame image in the monitoring video is obtained; based on OCR recognition of the time and kilometer mark information in the monitoring video, any video frame image is associated with the LKJ information. The monitoring video and LKJ are consistent in time, and then the target time when the locomotive crew has standardized behavior is determined according to the needs, and the corresponding target monitoring video is intercepted.

[0144] According to the acquisition time, all video frame images within the monitoring period are determined as target monitoring videos.

[0145] Optionally, the second determining unit 1205 is specifically configured to:

[0146] For any video frame image in the target surveillance video, the arm area and the hand area in the video frame image are obtained, and the arm area and the hand area belong to the key areas of the human body;

[0147] Determine whether the aspect ratio of the arm region is greater than an aspect ratio threshold, and whether the ordinate of the center point of the hand region is not greater than the ordinate of the center point of the arm region;

[0148] If so, it is determined that the standard behavior of locomotive crew members is hand gesture behavior.

[0149] Optionally, the device is also used for:

[0150] Before determining that the behavior of the locomotive crew member is a hand gesture behavior, a locomotive crew member region in the video frame image is obtained, and the locomotive crew member region belongs to a key region of the human body;

[0151] respectively calculating a first degree of overlap between the arm region and the hand region, and a second degree of overlap between the arm region and the locomotive crew region;

[0152] It is determined that the first overlap degree and the second overlap degree are respectively greater than an overlap degree threshold.

[0153] Optionally, the second determining unit 1205 is specifically configured to:

[0154] For any video frame image in the target surveillance video, obtain the back area, side window area and door area of ​​the locomotive crew in the video frame image;

[0155] Determine whether the relative distance between the back area of ​​the locomotive crew and the side window area or the door area is not greater than the distance threshold. The back area of ​​the locomotive crew belongs to the key area of ​​the human body, and the side window area and the door area belong to the key area of ​​the cab. Regarding the setting of the distance threshold, the distance threshold can use a fixed value; considering that different monitoring video frame sizes may not be fixed values; the distance threshold can be defined according to the proportion or by calculating the distance between the seat and the side window or door.

[0156] If so, it is determined that the standard behavior of the locomotive crew is the rear lookout behavior.

[0157] Optionally, the second determining unit 1205 is specifically configured to:

[0158] For any video frame image in the target monitoring video, obtain the height of the locomotive crew member in the video frame image;

[0159] Determine whether the height of the locomotive crew is greater than the height threshold;

[0160] If so, it is determined that the standard behavior of the locomotive crew is standing up.

[0161] Optionally, the second determining unit 1205 is specifically configured to:

[0162] Acquire the head area of ​​the locomotive crew in the video frame image;

[0163] Determine the area of ​​the head region and the area of ​​the video frame image;

[0164] Calculate the area ratio of the head region to the video frame image;

[0165] Determine the interval to which the area ratio belongs; determine the reference height based on the door, the side window and the seat. Set a piecewise function based on the reference height, and determine the segmented interval to which the area ratio belongs;

[0166] The height threshold corresponding to the interval is used as the height threshold used for the video frame image;

[0167] It is determined that the height of the locomotive crew member is greater than the height threshold.

[0168] Based on the same concept, an electronic device is also provided in the embodiment of the present application, such as Fig.13 As shown, the electronic device mainly includes: a processor 1301, a memory 1302 and a communication bus 1303, wherein the processor 1301 and the memory 1302 communicate with each other through the communication bus 1303. The memory 1302 stores a program executable by the processor 1301, and the processor 1301 executes the program stored in the memory 1302 to implement the following steps:

[0169] Obtain monitoring videos of locomotive crew members during locomotive operation, and obtain target moments when locomotive crew members are required to have standardized behaviors from train operation monitoring information;

[0170] Determine the monitoring period corresponding to the target time;

[0171] Intercepting a target surveillance video corresponding to the surveillance period from the surveillance video;

[0172] Detect key areas in each video frame image in the target monitoring video, including key areas of the human body and key areas of the cab;

[0173] The standard behaviors of locomotive crew members are determined based on key areas. Standard behaviors are prescribed behaviors that locomotive crew members must perform during crew operations. Standard behaviors include hand gestures, rear lookout, and standing up.

[0174] The communication bus 1303 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 1303 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0175] The memory 1302 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Alternatively, the memory may also be at least one storage device located away from the processor 1301.

[0176] The above-mentioned processor 1301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0177] In another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed on a computer, the computer executes the method for analyzing the standardized behavior of locomotive crew members described in the above embodiment.

[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instruction is transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape, etc.), an optical medium (e.g., a DVD) or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0179] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0180] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for analyzing the normative behavior of a locomotive crew member, characterized in that: include: Obtaining monitoring video of the locomotive crew during the locomotive operation, and obtaining the target time when the locomotive crew needs to have standardized behavior from the train operation monitoring information; Determining a monitoring period corresponding to the target time; Intercepting a target surveillance video corresponding to the surveillance period from the surveillance video; Detecting key areas in each video frame image in the target monitoring video, wherein the key areas include a human body key area and a cab key area; Determine the standard behavior of the locomotive crew according to the key area, the standard behavior is the prescribed behavior that the locomotive crew must complete during the crew operation, and the standard behavior includes hand gesture behavior, rear lookout behavior, and standing behavior; Wherein, determining the standard behavior of the locomotive crew according to the key area includes: For any video frame image in the target surveillance video, an arm region and a hand region in the video frame image are obtained, wherein the arm region and the hand region belong to the key region of the human body; Determine whether the aspect ratio of the arm region is greater than the aspect ratio threshold, and whether the ordinate of the center point of the hand region is not greater than the ordinate of the center point of the arm region If yes, determining that the standard behavior of the locomotive crew is the hand gesture behavior; Before determining that the behavior of the locomotive crew is the hand gesture behavior, the method further includes: Acquire a locomotive crew region in the video frame image, where the locomotive crew region belongs to the key human body region; respectively calculating a first degree of overlap between the arm region and the hand region, and a second degree of overlap between the arm region and the locomotive crew region; It is determined that the first overlap degree and the second overlap degree are respectively greater than an overlap degree threshold.

2. The method according to claim 1, characterized in that Intercepting a target surveillance video corresponding to the surveillance period from the surveillance video includes: Based on OCR recognition, obtain the acquisition time of each video frame image in the surveillance video, and the OCR recognition content includes but is not limited to time and kilometer mark; According to the acquisition time, all video frame images within the monitoring period are determined to be the target monitoring video.

3. The method according to claim 1, characterized in that Determining the standard behavior of the locomotive crew according to the key area includes: For any video frame image in the target surveillance video, obtaining the back area, side window area and door area of ​​the locomotive crew in the video frame image; Determine whether the relative distance between the back area of ​​the locomotive crew and the side window area or the door area is not greater than a distance threshold, the back area of ​​the locomotive crew belongs to the key area of ​​the human body, and the side window area and the door area belong to the key area of ​​the cab; If so, determine that the standard behavior of the locomotive crew is the rear lookout behavior.

4. The method according to claim 1, characterized in that: Determining the standard behavior of the locomotive crew according to the key area includes: For any video frame image in the target monitoring video, obtaining the height of the locomotive crew member in the video frame image; Determining whether the height of the locomotive crew member is greater than a height threshold; If so, determine that the standard behavior of the locomotive crew is the standing up behavior.

5. The method according to claim 4, characterized in that Determining whether the height of the locomotive crew member is greater than a height threshold includes: Acquire the head area of ​​the locomotive crew in the video frame image; Determining the area of ​​the head region and the area of ​​the video frame image; Calculating an area ratio of the head region relative to the video frame image; determining the interval to which the area ratio belongs; Using the height threshold corresponding to the interval as the height threshold used by the video frame image; It is determined that the height of the locomotive crew member is greater than the height threshold.

6. A device for analyzing the standard behavior of a locomotive crew, characterized in that: include: An acquisition unit, used to acquire monitoring video of the locomotive crew during the operation of the locomotive, and to acquire a target time when the locomotive crew needs to have a standardized behavior from the train operation monitoring information; A first determining unit, configured to determine a monitoring period corresponding to the target time; An interception unit, used to intercept a target surveillance video corresponding to the surveillance period from the surveillance video; A detection unit, used to detect key areas in each video frame image in the target monitoring video, wherein the key areas include a human body key area and a cab key area; A second determining unit is used to determine the standard behavior of the locomotive crew according to the key area, wherein the standard behavior is a prescribed behavior that the locomotive crew must complete during the crew operation, and the standard behavior includes hand gesture behavior, rear lookout behavior or standing behavior; Wherein, the second determining unit is used for: For any video frame image in the target surveillance video, an arm region and a hand region in the video frame image are obtained, wherein the arm region and the hand region belong to the key region of the human body; Determine whether the aspect ratio of the arm region is greater than the aspect ratio threshold, and whether the ordinate of the center point of the hand region is not greater than the ordinate of the center point of the arm region If yes, determining that the standard behavior of the locomotive crew is the hand gesture behavior; Wherein, the device is also used for: Acquire a locomotive crew region in the video frame image, where the locomotive crew region belongs to the key human body region; respectively calculating a first degree of overlap between the arm region and the hand region, and a second degree of overlap between the arm region and the locomotive crew region; It is determined that the first overlap degree and the second overlap degree are respectively greater than an overlap degree threshold.

7. An electronic device, characterized in that: include: A processor, a memory and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to execute the program stored in the memory to implement the method for analyzing the standardized behavior of locomotive crew members described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for analyzing the standardized behavior of a locomotive crew member according to any one of claims 1 to 5 is implemented.

Images