Urban rail transit finger mouth calling system based on posture recognition

Abstract

The invention discloses an urban rail transit finger mouth calling system based on posture recognition, and the system comprises an acquisition and analysis module which is used for obtaining an A-type instruction representing indication operation and mouth calling content, and a B-type instruction representing actuator operation and mouth calling content, and for the real-time A-type instruction of a user, a gesture recognition module which is used for performing gesture recognition on the user; acquiring a collection point representing the time when the finger of the user appears in the target area, and marking a time point when the user instruction operation is recognized to be correct as a capture moment within a standard duration starting from the collection point, the standard duration referring to a duration required for demonstrating the instruction operation; relates to the technical field of urban rail transit, and aims to effectively screen irrelevant hand actions in finger mouth calling commands by separately monitoring indication operation of a driver and operation of an actuator in different modes, prevent the irrelevant hand actions from influencing judgment of the finger mouth calling commands and improve the accuracy of action recognition.

Description

Technical Field

[0001] The invention belongs to the technical field of urban rail transit, and in particular is an urban rail transit finger-mouth calling system based on gesture recognition. Background Art

[0002] Finger-to-mouth communication in urban rail transit is a method of operation in which workers point their fingers at relevant equipment or signs and loudly call out relevant content during urban rail operations to ensure accurate and safe operations.

[0003] The existing technology usually requires real-time monitoring of the finger-mouth commands made by the driver during the entire train operation. Continuous operation will not only cause unnecessary energy consumption, but may also accelerate equipment aging due to long-term operation. In the monitoring process, it is easy to misjudge the finger movements in the finger-mouth commands, affecting the judgment results. At the same time, there will be a problem that the finger movements and the oral content are correct but there is a delay between the two. The coherence between the finger movements and the oral content is not considered, and the standardization of the finger-mouth commands is judged incorrectly, resulting in poor recognition accuracy.

[0004] In order to solve the above problems, the present invention proposes a solution. Summary of the invention

[0005] The present invention aims to solve the problems raised in the above background technology; to this end, the present invention proposes a finger-to-mouth calling system for urban rail transit based on gesture recognition, comprising: A collection and analysis module, used to obtain Class A instructions representing instruction operations and spoken content, and Class B instructions representing actuator operations and spoken content; And for the user's real-time Class A instruction, a collection point indicating the time when the user's finger appears in the target area is obtained, and the time point when the user's instruction operation is recognized to be correct is marked as the capture moment within the standard time length starting from the collection point, and the standard time length refers to the time length required for demonstrating the instruction operation; For Class B instructions, when it is detected that the digital mark of the execution signal generated by the sensor is consistent with the digital mark of the corresponding actuator operation in the storage library, the execution signal generation time is obtained; The duration between the capture moment and the output time is marked as the interval duration SC1; the duration between the execution signal generation time and the output time is marked as the interval duration SC2, and the output time refers to the time point when the driver's spoken content is determined to be consistent with the standard text; by comparing the interval duration SC1 and the interval duration SC2 with the preset value AJ respectively, it is used to determine whether Class A instructions and Class B instructions meet the standard.

[0006] Preferably, it also includes an acquisition module for acquiring the number of all stations on the fixed rail line from the subway official website.

[0007] Preferably, a division module is also included, which is used to divide the monitoring section and the collection interval according to the number of sites, and the specific division method is: During the operation of the fixed rail line, the total time taken for the subway to depart from the first station until it leaves the next station is marked as a monitoring segment. In the same way as above, the number of monitoring segments on the fixed rail line is obtained according to the number of stations. Select any monitoring segment, obtain all finger-mouth commands in the monitoring segment, select any finger-mouth command, obtain the standard time point for making the finger-mouth command, and set the collection time and end time before and after the standard time point, respectively. The collection time and end time have the same time interval from the standard time point, and the time period from the collection time to the end time is uniformly marked as the collection interval; In the same manner as above, the collection interval of each finger-mouth command is obtained one by one.

[0008] Preferably, a pre-processing module is also included, which is used to divide the finger-mouth commands into Class A commands and Class B commands.

[0009] Preferably, for the indicated operation in the Class A instruction, the driver's hand area is photographed by a high-definition camera to obtain an irregular image obtained after the photographing, and then the center points of the wrist joint and the elbow joint in the irregular image are determined by a detection algorithm; Select any indication operation, obtain the indication target in the cab under the indication operation, and divide the target area according to the indication target. Establish a three-dimensional rectangular coordinate system in the cab with a preset point as the origin. By obtaining the coordinates of all pixel points in the target area, the average value of all coordinates is directly calculated, and then the center point of the target area is determined; In the same manner as above, the center point of the target area under each indication operation is obtained one by one, and an infrared sensor is installed at the position of the center point, and each infrared sensor is marked with a different letter; The driver makes corresponding demonstration instruction operations in advance according to each instruction operation, obtains the standard duration of each demonstration instruction operation through a timer according to the synchronization principle of the instruction operation and the oral content, and stores the standard duration in the first database in a one-to-one correspondence with the instruction operation.

[0010] Preferably, for the actuator operation in the Class B instruction, a sensor is pre-set on each actuator, and the execution signal output by each sensor is marked with a different digital tag and stored in the storage library; For each actuator operation or indication operation, the driver uses a voice recognition device to input the oral content corresponding to the actuator operation or indication operation, and then converts the oral content into text information, which is recalibrated into standard text, and the standard text is stored in the second database in a one-to-one correspondence with the actuator operation or indication operation.

[0011] Preferably, the specific analysis method in the acquisition and analysis module is as follows: Select a finger-voice command in any collection interval. For Class A commands, when the driver's finger points to the target area, the infrared sensor marked with the letter corresponding to the target area will detect a signal change, then it is determined that the driver's finger action occurs in the target area, and the time point of the first detected signal change is marked as the collection point; Then, the center points of the driver's wrist joint and elbow joint are obtained, and the two are connected to obtain a baseline. With the center point of the wrist joint as the endpoint, the baseline is infinitely extended toward the finger direction. On the edge contour line of the target area, the position of the point farthest from the indication point in a straight line is obtained, and it is marked as a fixed point. The straight-line distance ZX1 between the fixed point and the indication point is obtained; Get the standard duration of the corresponding indication operation. Starting from the acquisition point, obtain the vertical distance CZ from the indication point of the target area to the baseline within the standard duration. When CZ≤ZX1 is satisfied for n consecutive cycles, the judgment result is output, that is, the indication operation is correct, and the moment of outputting the judgment result is marked as the capture moment.

[0012] Preferably, for Class B instructions, when the driver operates the actuator, the sensor on the corresponding actuator will generate an execution signal, obtain the corresponding digital mark on the execution signal and the generation time of the execution signal, recalibrate the obtained digital mark as a mark to be identified, extract the digital mark under the corresponding actuator operation in the repository, recalibrate it as a specified mark, and compare the mark to be identified with the specified mark. When the mark to be identified does not completely match the specified mark or the execution signal is not detected, it is determined that the actuator operation is incorrect. When the mark to be identified completely matches the specified mark, it is determined that the actuator operation is correct.

[0013] Preferably, the capture moment, the interval between the execution signal generation time and the output time are marked as SC1 and SC2 respectively; When SC1≤AJ, it is determined that the Class A instruction completed by the driver meets the standard; when SC2≤AJ, it is determined that the Class B instruction completed by the driver meets the standard; conversely, when SC1>AJ, it is determined that the Class A instruction completed by the driver does not meet the standard; when SC2>AJ, it is determined that the Class B instruction completed by the driver does not meet the standard; The finger-mouth commands in the remaining collection intervals are processed in the same manner as above, and the finger-mouth commands that do not meet the standard are marked as non-standard commands.

[0014] Preferably, the method further includes a post-processing module, which is used to process irregular commands. The specific processing method is: On a fixed track running line, the total number A of finger-mouth commands is obtained, and for each finger-mouth command, the total number of times ZCSi marked as irregular commands by n drivers under the finger-mouth command is obtained, i=1, ..., A; When ZCSi≥Y, it is determined that the probability of the finger-mouth command not meeting the standard is high, where Y is a preset value.

[0015] Compared with the prior art, the present invention has the following beneficial effects: Finger-to-mouth commands are divided into Class A commands and Class B commands. Class A commands are instruction operations and oral contents, while Class B commands are actuator operations and oral contents. By using different methods to monitor the driver's instruction operations and actuator operations separately, irrelevant hand movements in finger-to-mouth commands can be effectively screened out, avoiding irrelevant hand movements from affecting the judgment of finger-to-mouth commands, and improving the accuracy of action recognition. At the same time, in the same finger-mouth command, when it is recognized that the driver's oral content is correct and corresponds to the indicated operation or actuator operation one by one, the standardization of the finger-mouth command is judged by analyzing the interval between the oral content and the indicated operation or actuator operation. For multiple drivers on a fixed track line, the total number of times non-standard commands are marked under the same finger-mouth command is counted, so that finger-mouth commands with a higher probability of non-standard commands can be obtained, which is convenient for subsequent targeted training. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the principle framework of the present invention; Figure 2 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0017] The technical scheme of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1 See also Figure 1 , 2,This application provides a finger-to-mouth calling system for urban rail transit based on gesture recognition, including; The acquisition module acquires the official website of the city's subway, which records in detail the route map, station list, transfer guide and real-time train arrival time information. Based on the above information, the number of all stations on the fixed section track line is acquired and the number of stations is transmitted to the division module; The division module receives the number of sites and divides the monitoring section and collection interval according to the number of sites. The specific division method is as follows: During the operation of the fixed rail line, the total time taken for the subway to depart from the first station until it leaves the next station is marked as a monitoring segment. In the same way as above, the number of monitoring segments on the fixed rail line is obtained according to the number of stations. Select any monitoring section, obtain all finger-to-mouth commands in the monitoring section, where finger-to-mouth commands refer to finger movements and corresponding oral contents, select any finger-to-mouth command, obtain the standard time point for making the finger-to-mouth command, and set the collection time and end time respectively before and after the standard time point, where the collection time and the end time have the same time interval from the standard time point, and the time period from the collection time to the end time is uniformly marked as the collection interval, for each collection interval, when the time reaches the preset value near the collection time, the buzzer is started, and the driver is reminded to prepare through the buzzer, and when the collection time is reached, the monitoring equipment is immediately started to start comprehensive monitoring work, and once the time reaches the end time, the monitoring equipment is turned off and the monitoring task is stopped, and the monitoring equipment does not need to be turned on in real time, but only needs to be enabled during a specific period of time, which can not only accurately obtain key information, but also reduce the operating burden; In the same manner as above, the collection interval of each finger-mouth command is obtained one by one; Pre-processing module: monitoring equipment is installed at a suitable location in the subway cab in advance. The monitoring equipment includes high-definition cameras and voice recognition equipment. The high-definition cameras ensure that the driver's hand area can be clearly photographed. The voice recognition equipment uses high-performance products to ensure that the driver's oral content can be accurately identified and quickly converted into accurate text information. Before taking up their posts, the driver shall establish standardized standards for different types of finger-mouth commands in accordance with the standardization principle of finger-mouth commands. The specific processing methods are as follows: Finger-to-mouth commands are divided into Class A commands and Class B commands in advance. Class A commands are instruction operations and verbal commands. For example, when a subway passes a switch, the finger indicates the switch position and verbally calls out: "The switch is open straight / sideways, correct." Class B commands are actuator operations and verbal commands. For example, when a subway arrives at a station, the door control button is operated and verbally calls out: "The door closing button is pressed, and the door is closing." Here, the actuator refers to the instruments and equipment used in Class B commands, including control buttons, operating handles, etc. For the instruction operation in the Class A instruction, the driver's hand area is photographed by a high-definition camera to obtain an irregular image obtained after the photographing, and then the center points of the wrist joint and the elbow joint in the irregular image are determined by a detection algorithm. The method of determining the center point of the irregular image is an existing technology, including an edge detection method, a self-centroid method, and a segmentation method, which is not described in detail in this application document; Taking one of the indication operations as an example, the indication target in the cab under the indication operation is obtained, and the target area is divided according to the indication target. The target area is manually preset and it is necessary to ensure that the indication target is located in the target area. A three-dimensional rectangular coordinate system is established in the cab with the preset point as the origin. By obtaining the coordinates of all pixel points in the target area, the average value of all coordinates is directly calculated, and then the center point of the target area is determined; In the same manner as above, the center point of the target area under each indication operation is obtained one by one, and it is re-marked as an indication point, and an infrared sensor is installed at the position of the indication point, and each infrared sensor is marked with a different letter. The infrared sensor can be an infrared sensor of model JHW2-TOC. Of course, those skilled in the art can choose other models according to needs; The driver makes a corresponding demonstration instruction operation in advance according to each instruction operation, obtains the standard duration of each demonstration instruction operation through a timer according to the synchronization principle of the instruction operation and the oral call content, and stores the standard duration in a first database in a one-to-one correspondence with the instruction operation; For the actuator operation in the Class B instruction, a sensor is pre-set on each actuator, and the execution signal output by each sensor is marked with a different digital tag and stored in the memory bank; For each actuator operation or indication operation, the driver uses the voice recognition device to input the spoken content corresponding to the actuator operation or indication operation, and then converts the spoken content into text information, recalibrates it into standard text, and stores the standard text in a second database in a one-to-one correspondence with the actuator operation or indication operation; The collection and analysis module collects and analyzes the driver's finger movements and spoken words in real time for Class A and Class B instructions during subway operation. The specific analysis method is as follows: Taking the finger-to-mouth command in one of the collection intervals as an example, for Class A commands, when the driver's finger points to the target area, the infrared sensor marked with letters corresponding to the target area will detect a signal change. The signal change indicates signal interruption or weakening. It is determined that the driver's finger action occurs in the target area, and the time point of the first detected signal change is marked as the collection point. Then, the center points of the driver's wrist joint and elbow joint are obtained, and the two are connected to obtain a baseline. With the center point of the wrist joint as the endpoint, the baseline is infinitely extended toward the finger direction. On the edge contour line of the target area, the position of the point farthest from the indication point in a straight line is obtained, and it is marked as a fixed point. The straight-line distance ZX1 between the fixed point and the indication point is obtained; Get the standard duration of the corresponding indication operation. Starting from the acquisition point, obtain the vertical distance CZ from the indication point of the target area to the baseline within the standard duration. When CZ≤ZX1 is satisfied for n consecutive cycles, the judgment result is output, that is, the indication operation is correct, and the moment of outputting the judgment result is marked as the capture moment. Then, the driver's spoken words are recognized by the voice recognition device and converted into text information, the standard text corresponding to the indicated operation in the second database is obtained, and the text information is compared with the standard text; When the text information is completely consistent with the standard text, the output time of the comparison result is obtained, and the interval length SC1 between the capture time and the output time is obtained; When SC1≤AJ, it is determined that the Class A instruction completed by the driver meets the standard. Conversely, when SC1>AJ, it is determined that the Class A instruction completed by the driver does not meet the standard. For Class B instructions, when the driver is operating the actuator, the sensor on the corresponding actuator will generate an execution signal, obtain the corresponding digital mark on the execution signal and the generation time of the execution signal, recalibrate the obtained digital mark as a mark to be identified, extract the digital mark under the corresponding actuator operation in the storage library, recalibrate it as a designated mark, compare the mark to be identified with the designated mark, and when the mark to be identified does not completely match the designated mark, it is determined that the actuator operation is wrong. When the execution signal is not detected, it is also determined that the actuator operation is wrong. When the mark to be identified completely matches the designated mark, it is determined that the actuator operation is correct. Then, the driver's spoken words are recognized by the voice recognition device and converted into text information, the standard text corresponding to the actuator operation in the second database is obtained, and the text information is compared with the standard text; When the text information is completely consistent with the standard text, the output time of the comparison result is obtained, and the interval length SC2 between the execution signal generation time and the output time is obtained; When SC2≤AJ, it is determined that the Class B instruction completed by the driver meets the standard. Conversely, when SC2>AJ, it is determined that the Class B instruction completed by the driver does not meet the standard, where AJ is a manually preset time threshold; The finger-mouth commands in the remaining collection intervals are processed in the same manner as above, and the finger-mouth commands that do not meet the standard are marked as non-standard commands.

[0019] Embodiment 2 In the specific implementation process, this embodiment is different from the first embodiment in that: The post-processing module receives irregular commands, obtains finger-mouth commands with a high probability of irregular commands, and processes them. The specific processing method is as follows: On a fixed track running line, the total number A of finger-mouth commands is obtained, and for each finger-mouth command, the total number of times ZCSi marked as irregular commands by n drivers under the finger-mouth command is obtained, i=1, ..., A; When ZCSi≥Y, it is determined that the probability of non-standard finger-mouth commands is high, and the finger-mouth commands that meet the above conditions need to be trained intensively later, where Y is a preset value.

[0020] Some of the data in the above formulas are dimensionless for numerical calculations, and the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0021] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. The finger-to-mouth call system for urban rail transit based on gesture recognition is characterized by: include: A collection and analysis module, used to obtain Class A instructions representing instruction operations and spoken content, and Class B instructions representing actuator operations and spoken content; And for the user's real-time Class A instruction, a collection point indicating the time when the user's finger appears in the target area is obtained, and the time point when the user's instruction operation is recognized to be correct is marked as the capture moment within the standard time length starting from the collection point, and the standard time length refers to the time length required for demonstrating the instruction operation; For Class B instructions, when it is detected that the digital mark of the execution signal generated by the sensor is consistent with the digital mark of the corresponding actuator operation in the storage library, the execution signal generation time is obtained; The duration between the capture moment and the output time is marked as the interval duration SC1; the duration between the execution signal generation time and the output time is marked as the interval duration SC2, and the output time refers to the time point when the driver's spoken content is determined to be consistent with the standard text; by comparing the interval duration SC1 and the interval duration SC2 with the preset value AJ respectively, it is used to determine whether Class A instructions and Class B instructions meet the standard.

2. The urban rail transit finger-to-mouth calling system based on gesture recognition according to claim 1 is characterized in that: It also includes an acquisition module for acquiring the number of all stations on the fixed track line from the subway official website.

3. The urban rail transit finger-to-mouth calling system based on gesture recognition according to claim 2 is characterized in that: It also includes a division module, which is used to divide the monitoring section and the collection interval according to the number of sites. The specific division method is: During the operation of the fixed rail line, the total time taken for the subway to depart from the first station until it leaves the next station is marked as a monitoring segment. In the same way as above, the number of monitoring segments on the fixed rail line is obtained according to the number of stations. Select any monitoring segment, obtain all finger-mouth commands in the monitoring segment, select any finger-mouth command, obtain the standard time point for making the finger-mouth command, and set the collection time and end time before and after the standard time point, respectively. The collection time and end time have the same time interval from the standard time point, and the time period from the collection time to the end time is uniformly marked as the collection interval; In the same manner as above, the collection interval of each finger-mouth command is obtained one by one.

4. The urban rail transit finger-to-mouth calling system based on gesture recognition according to claim 1 is characterized in that: It also includes a pre-processing module for dividing finger-mouth commands into Class A commands and Class B commands.

5. The urban rail transit finger-to-mouth calling system based on gesture recognition according to claim 4 is characterized in that: For the instruction operation in the Class A instruction, the driver's hand area is photographed by a high-definition camera to obtain the irregular image obtained after the shooting, and then the center points of the wrist joint and elbow joint in the irregular image are determined by the detection algorithm; Select any indication operation, obtain the indication target in the cab under the indication operation, and divide the target area according to the indication target. Establish a three-dimensional rectangular coordinate system in the cab with a preset point as the origin. By obtaining the coordinates of all pixel points in the target area, the average value of all coordinates is directly calculated, and then the center point of the target area is determined; In the same manner as above, the center point of the target area under each indication operation is obtained one by one, and an infrared sensor is installed at the position of the center point, and each infrared sensor is marked with a different letter; The driver makes corresponding demonstration instruction operations in advance according to each instruction operation, obtains the standard duration of each demonstration instruction operation through a timer according to the synchronization principle of the instruction operation and the oral content, and stores the standard duration in the first database in a one-to-one correspondence with the instruction operation.

6. The urban rail transit finger-mouth calling system based on gesture recognition according to claim 5 is characterized in that: For the actuator operation in the Class B instruction, a sensor is pre-set on each actuator, and the execution signal output by each sensor is marked with a different digital tag and stored in the memory bank; For each actuator operation or indication operation, the driver uses a voice recognition device to input the oral content corresponding to the actuator operation or indication operation, and then converts the oral content into text information, which is recalibrated into standard text, and the standard text is stored in the second database in a one-to-one correspondence with the actuator operation or indication operation.

7. The urban rail transit finger-mouth calling system based on gesture recognition according to claim 1 is characterized in that: The specific analysis method in the acquisition and analysis module is as follows: Select a finger-voice command in any collection interval. For Class A commands, when the driver's finger points to the target area, the infrared sensor marked with the letter corresponding to the target area will detect a signal change, then it is determined that the driver's finger action occurs in the target area, and the time point of the first detected signal change is marked as the collection point; Then, the center points of the driver's wrist joint and elbow joint are obtained, and the two are connected to obtain a baseline. With the center point of the wrist joint as the endpoint, the baseline is infinitely extended toward the finger direction. On the edge contour line of the target area, the position of the point farthest from the indication point in a straight line is obtained, and it is marked as a fixed point. The straight-line distance ZX1 between the fixed point and the indication point is obtained; Get the standard duration of the corresponding indication operation. Starting from the acquisition point, obtain the vertical distance CZ from the indication point of the target area to the baseline within the standard duration. When CZ≤ZX1 is satisfied for n consecutive cycles, the judgment result is output, that is, the indication operation is correct, and the moment of outputting the judgment result is marked as the capture moment.

8. The urban rail transit finger-mouth calling system based on gesture recognition according to claim 7 is characterized in that: For Class B instructions, when the driver operates the actuator, the sensor on the corresponding actuator will generate an execution signal, obtain the corresponding digital mark on the execution signal and the generation time of the execution signal, recalibrate the obtained digital mark as a mark to be identified, extract the digital mark under the corresponding actuator operation in the repository, recalibrate it as a specified mark, and compare the mark to be identified with the specified mark. When the mark to be identified does not completely match the specified mark or the execution signal is not detected, it is determined that the actuator operation is wrong. When the mark to be identified completely matches the specified mark, it is determined that the actuator operation is correct.

9. The urban rail transit finger-mouth calling system based on gesture recognition according to claim 8 is characterized in that: The capture moment, the execution signal generation time and the output time are marked as SC1 and SC2 respectively; When SC1≤AJ, it is determined that the Class A instruction completed by the driver meets the standard; when SC2≤AJ, it is determined that the Class B instruction completed by the driver meets the standard; conversely, when SC1>AJ, it is determined that the Class A instruction completed by the driver does not meet the standard; when SC2>AJ, it is determined that the Class B instruction completed by the driver does not meet the standard; The finger-mouth commands in the remaining collection intervals are processed in the same manner as above, and the finger-mouth commands that do not meet the standard are marked as non-standard commands.

10. The urban rail transit finger-mouth calling system based on gesture recognition according to claim 9 is characterized in that: It also includes a post-processing module for processing irregular commands. The specific processing method is as follows: On a fixed track running line, the total number A of finger-mouth commands is obtained, and for each finger-mouth command, the total number of times ZCSi marked as irregular commands by n drivers under the finger-mouth command is obtained, i=1, ..., A; When ZCSi≥Y, it is determined that the probability of the finger-mouth command not meeting the standard is high, where Y is a preset value.

Images