An intelligent management platform for public transportation services

Abstract

The present invention discloses an intelligent management platform for public transportation services, which relates to the technical field of intelligent public transportation monitoring. It conducts real-time monitoring and management of abnormalities caused by physical discomfort of passengers riding on public transportation, and adopts a monitoring method combining video and physiological signals. By analyzing the video images of passengers and the physiological signals of passengers, it can timely and accurately detect the physical discomfort of passengers, and ensure that timely and effective help is given to passengers when they have various conditions. In terms of hardware, it is only necessary to arrange physiological signal monitoring pads on special seats, and then use the existing cameras on the bus to implement it. There is no need to make major changes to the bus, and it is easy to implement; in terms of software, a neural network combined with 2D / 3D convolution is used to improve the performance and efficiency of abnormal behavior recognition; the combination of video image analysis and physiological signal analysis, as well as the manual confirmation of passenger abnormality recognition and judgment methods are adopted to minimize misjudgment.

Description

Technical Field

[0001] The present invention relates to the technical field of public transportation intelligent monitoring, and in particular to an intelligent management platform for public transportation services. Background Art

[0002] As the process of urbanization continues to accelerate, buses have become an indispensable part of our lives. In order to ensure the safety of people during bus driving, monitoring equipment such as cameras are usually installed on buses to monitor the situation inside the bus in real time.

[0003] At present, the research on personnel safety monitoring of buses often focuses on the monitoring of drivers, such as monitoring whether the driver smokes, whether he takes his hands off the steering wheel, whether he leaves the driving seat during driving, whether he drives fatigued and other unsafe driving behaviors; monitoring whether the driver has sudden situations such as heart disease that make him unable to drive normally; and taking a series of safety measures such as alarm, automatic braking, automatic speed limit and parking when abnormalities of the driver are detected.

[0004] However, due to the large number of passengers on the bus and the closed interior of the bus, special passengers such as the elderly and pregnant women may experience sudden physical discomfort while riding the bus, or even sudden myocardial infarction in the elderly or pregnant women about to give birth, which require emergency treatment. In these cases, the driver cannot observe them while driving the vehicle, and the surrounding passengers can only notice when the unwell passengers make major movements, such as falling to the ground or asking for help. If an elderly person goes into shock in his seat, it will not be discovered at all.

[0005] Therefore, there is an urgent need for an intelligent management platform for public transportation services to conduct intelligent supervision of bus passengers, ensure timely and effective assistance when passengers encounter various situations, and ensure the safety of all personnel during the operation of the bus. Summary of the invention

[0006] The present invention provides a public transportation business intelligent management platform to realize intelligent supervision of bus passengers and ensure the safety of all personnel during the driving of the bus.

[0007] To this end, the present invention provides the following technical solutions:

[0008] The present invention provides a public transportation business intelligent management platform, which mainly includes: a passenger monitoring and management system for monitoring abnormalities caused by physical discomfort of multiple passengers; the passenger monitoring and management system includes: a platform end and a bus end; wherein:

[0009] The platform includes:

[0010] A video information receiving unit for receiving video information transmitted from a bus terminal;

[0011] A physiological information receiving unit for receiving physiological information transmitted by the bus terminal;

[0012] A data preprocessing unit for preprocessing the video information obtained by the video information receiving unit and the physiological information obtained by the physiological information receiving unit;

[0013] A video data analysis unit for receiving the sparsely processed data of the picture frames in the video stream transmitted by the data preprocessing unit, segmenting each passenger in the image using an image segmentation model for the sparsely processed data of the picture frames in the video stream, and then distinguishing the abnormal behavior and normal behavior of the passengers using an abnormal analysis model based on deep learning for each segmented sub-image, so as to obtain the distinction result of the normal behavior and abnormal behavior of the passengers when taking the bus;

[0014] A physiological information analysis unit for analyzing the preprocessed physiological information signal, wherein the physiological information analysis unit extracts the BCG signal, respiratory information and body movement information therein and analyzes them respectively to obtain the human heart rate value and respiratory frequency and determine whether the passenger is in the seat; and judges whether the analyzed physiological information is within the safety threshold. If so, the analysis result is normal; if not, the analysis result is abnormal;

[0015] An abnormality determination unit for determining whether the passenger has abnormality caused by physical discomfort based on the analysis results of the video data analysis unit and the physiological information data analysis unit;

[0016] When the abnormality determination unit determines that a passenger on the bus is abnormal, an alarm is sent to the monitoring personnel in the central monitoring room, and the monitoring personnel further confirms based on the video and triggers the alarm unit of the dispatching unit after confirmation;

[0017] A dispatching unit for dispatching an ambulance or an idle public bus to the location of the bus where the passenger is located, so as to provide medical treatment to the passenger and transfer other passengers;

[0018] The bus terminal includes:

[0019] a video data acquisition and transmission unit for acquiring and transmitting video data of passengers in a bus compartment using a camera, and,

[0020] A physiological information collection and transmission unit is used to collect and transmit the physiological information of passengers on bus seats using physiological information monitoring pads arranged on each seat.

[0021] Furthermore, the image segmentation model adopts a mask-based regional convolutional neural network.

[0022] Furthermore, the anomaly analysis model is a deep learning model formed by combining a 2D convolutional subnetwork and a 3D convolutional subnetwork through feature splicing;

[0023] The 2D convolution subnetwork is composed of four Ghost-Bottleneck blocks, which are stacked by two GhostModule modules and a depth-separable convolution module. The GhostModule is divided into main convolution operation and simple transformation operation. The main convolution generates a feature map, and then the simple transformation operation is used to transform the feature map generated by the main convolution. Finally, the two parts of the feature map are spliced ​​together and input to the next GhostModule. The attention mechanism of the SE module is added to the fourth Ghost Bottleneck block.

[0024] The 3D convolution subnetwork adopts the structure of 3D-Shufflenetv2, which consists of 6 3D convolution modules. Each 3D convolution module is calculated by 1×1×1, 3×3×3 and 1×1×1 convolutions respectively, and finally merged together to the next 3D convolution module. Finally, the fully connected layer is used to classify abnormal and normal behaviors.

[0025] Furthermore, the physiological information monitoring pad is provided with a physiological information collecting device and an information transmission device for collecting physiological information inside its pad body;

[0026] The physiological information acquisition device includes a piezoelectric film sensor and a pressure sensor. The piezoelectric film sensor is used to detect weak change signals, based on which physiological signals can be obtained; the pressure sensor is used to detect whether there is anyone on the seat.

[0027] Furthermore, the physiological information includes respiratory rate and heart rate values.

[0028] Furthermore, the seats are reserved for the elderly, the weak, the sick, the disabled and pregnant women.

[0029] Furthermore, for the video image information, the preprocessing includes target area cropping and picture frame sparse processing. The target area is the area where the "old, weak, sick, disabled, and pregnant" seats are located in the bus compartment; the target area is cropped for each picture frame; the sparse processing refers to: the picture frame is preprocessed in two steps by sliding windows. The first step is to select 8 seconds as a segment, and after sparse sampling of 16 frames in a segment, these 16 frames are processed into 224×224 size picture frames. The second step is to convert the picture frame into a tensor format that the model can accept;

[0030] For physiological information, preprocessing includes denoising and segmentation. Denoising means removing power frequency interference and signal baseline to obtain original waveform data; segmentation means distinguishing the physiological signals of different passengers in the seat. The time node when the pressure sensor detects whether there is a passenger sitting on the seat is used as the basis. The physiological signals collected during the time period when a passenger is detected sitting on the seat and when a passenger leaves the seat are regarded as the physiological signals of one passenger.

[0031] Furthermore, the abnormality determination unit is specifically used for:

[0032] Matching video data and physiological information according to the acquisition time and seat number, the matched video data and physiological information are information of the same passenger;

[0033] If the analysis result of the video data analysis unit is that the passenger is abnormal, and the analysis result of the physiological information data analysis unit is that the passenger is abnormal, it is determined that the passenger is abnormal.

[0034] Furthermore, the safety threshold of heart rate is no more than 80 times / minute, and the safety threshold of respiratory rate is 12 times / minute-24 times / minute.

[0035] Advantages and positive effects of the present invention:

[0036] The bus service intelligent management platform in the present invention monitors and manages the abnormalities caused by physical discomfort of passengers riding the bus in real time, adopts a monitoring method combining video and physiological signals, and timely and accurately finds the physical discomfort of passengers by analyzing the video images and physiological signals of passengers, ensuring timely and effective help when passengers have various conditions, and ensuring the safety of all personnel during the driving of the bus. At the same time, the bus service intelligent management platform in the embodiment of the present invention, in terms of hardware, only needs to arrange physiological signal monitoring pads on the "old, weak, sick, disabled, pregnant" special seats, and then use the existing cameras on the bus to implement it, without making major changes to the bus, and is easy to implement; in terms of software, a neural network combined with 2D convolution and 3D convolution is used to greatly improve the performance and efficiency of abnormal behavior recognition; at the same time, a combination of video image analysis and physiological signal analysis, as well as a manually confirmed passenger abnormality recognition and judgment method are adopted to minimize misjudgment, thereby reducing the waste caused by resource scheduling after misjudgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 This is an architecture diagram of a public transportation business intelligent management platform in an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the structure of the passenger monitoring and management system (platform side and bus side) in an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of the network structure of the abnormal analysis model in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] like Figure 1 , which shows the architecture of a bus business intelligent management platform in an embodiment of the present invention. The bus business intelligent management platform can be deployed in the central monitoring room of a bus company, and can communicate with all buses registered with the bus company in real time to achieve intelligent supervision of all buses registered with the bus company, especially to achieve intelligent supervision of passengers on each bus.

[0044] The public transportation business intelligent management platform mainly includes: a passenger monitoring and management system for monitoring abnormalities caused by physical discomfort of multiple passengers. In other embodiments, the public transportation business intelligent management platform may also include other monitoring and management systems, such as a driver monitoring and management system for monitoring abnormalities caused by unsafe driving of the driver, and a second passenger monitoring and management system for monitoring abnormalities caused by other reasons such as fighting, falling, theft, etc. of multiple passengers. Both the driver monitoring and management system and the second passenger monitoring and management system can adopt existing systems in the prior art, which will not be repeated here.

[0045] Among them, Figure 2 As shown, the passenger monitoring and management system includes: a platform end 200 and a bus end 300; the passenger monitoring and management system of the platform end 200 can be deployed on a high-performance computer to efficiently complete data processing and calculation, and includes: a video information receiving unit 201, a physiological information receiving unit 202, a data preprocessing unit 203, a video data analysis unit 204, a physiological information analysis unit 205, an abnormality determination unit 206, an alarm unit 207 and a scheduling unit 208. Among them:

[0046] The video information receiving unit 201 is used to receive the video information transmitted by the bus terminal;

[0047] A physiological information receiving unit 202, used for receiving physiological information transmitted by the bus terminal;

[0048] The data preprocessing unit 203 is used to preprocess the video information obtained by the video information receiving unit 201 and the physiological information obtained by the physiological information receiving unit 203;

[0049] For video image information, the preprocessing operations include target area cropping and picture frame sparse processing. The target area is the area where the seats for the elderly, weak, sick, disabled, and pregnant women are located in the bus. The target area is cropped for each picture frame. The sparse processing refers to: preprocessing the picture frames in two steps by sliding windows. The first step is to select 8 seconds as a segment. After sparse sampling of 16 frames in a segment, these 16 frames are processed into 224×224 picture frames. The second step is to convert the picture frames into a tensor format that the model can accept.

[0050] For physiological information, the preprocessing operations include denoising and segmentation. Denoising means removing the power frequency interference and signal baseline to obtain the original waveform data. Specifically, a bandpass filter is used to preprocess the signal, and the signal envelope is extracted after removing some obvious noise interference, and a smoothing function is used to obtain a rough processed signal. Segmentation refers to distinguishing the physiological signals of different passengers in the seat. Specifically, based on the time node when the pressure sensor detects whether there is a passenger sitting in the seat, the physiological signals collected during the time period when a passenger is detected sitting in the seat and when a passenger leaves the seat are regarded as the physiological signals of one passenger.

[0051] The video data analysis unit 204 is used to receive the data after sparse processing of the picture frames in the video stream transmitted by the data preprocessing unit 203, and use the image segmentation model to segment each passenger in the image for the data after sparse processing of the picture frames in the video stream, and then use the abnormal analysis model to distinguish the abnormal behavior and normal behavior of the passengers for each segmented sub-image, so as to obtain the distinction result of the normal behavior and abnormal behavior of the passengers when taking the bus.

[0052] Among them, the image segmentation model can adopt the Mask-based Regional Convolutional Neural Network (Mask R-CNN). MASKR-CNN is an instance segmentation model and can be regarded as a faster R-CNN bounding box detection model with an additional mask branch, namely a small fully convolutional network (FCN).

[0053] At inference time, the mask branch is applied to each detected object to predict an instance-level foreground segmentation mask. During training, the mask branch is trained in parallel with the standard bounding box head in Faster R-CNN. In Mask R-CNN, both the last layer in the bounding box branch and the last layer in the mask branch contain class-specific (bus seats + passengers) parameters, which are used to perform bounding box classification and instance mask prediction for specific classes, respectively.

[0054] Each target bounding box segmented by the image segmentation model is cropped to obtain multiple sub-images, each of which includes a passenger sitting on a bus seat, and each sub-image is named according to the bus number and the seat number.

[0055] Among them, Figure 3 As shown in FIG. 1 , the anomaly analysis model is a deep learning model formed by combining 2D convolutional subnetwork and 3D convolutional subnetwork through feature concatenation. The input of the model is an RGB image frame. The anomaly analysis of bus passengers using the anomaly analysis model includes an inference phase and a training phase.

[0056] In the inference phase, the 2D convolutional subnetwork is composed of four Ghost-Bottleneck blocks, each of which is composed of two GhostModule modules and a depth-separable convolutional module. The GhostModule is divided into a main convolution operation and a simple transformation operation. The main convolution generates a feature map, and then the simple transformation operation is used to transform the feature map generated by the main convolution. The purpose is to enhance the features while reducing the amount of calculation. Finally, the two parts of the feature map are spliced ​​together and input to the next GhostModule. The SE module (Squeeze-and-Excitation) attention mechanism is added to the fourth Ghost Bottleneck block. The 2D convolutional subnetwork receives an input of size 3×224×224, where 3 is the number of channels of the RGB image frame, and 224×224 is the height and width of the image respectively. The output of the 2D convolutional subnetwork is 40×28×28. The feature map is stacked into a size of 16×40×28×28 in frame order and input into the 3D subnetwork. The 3D convolution subnetwork adopts the structure of 3D-Shufflenetv2, which consists of 6 3D convolution modules. Each 3D convolution module is calculated by 1×1×1, 3×3×3 and 1×1×1 convolutions respectively, and finally merged together to the next 3D convolution module. Finally, the fully connected layer is used to classify abnormal and normal behaviors.

[0057] During the training phase, historical video data of bus passengers is collected, and the video segments are annotated with normal and abnormal behaviors. The annotated video segments are used as data sets for training. In order to reduce the redundancy of most information between frames and minimize the loss of detection accuracy while ensuring the analysis speed, the image frames in the video are sparsely sampled and arranged in frame order. Finally, they are converted into tensor format and input into the abnormal analysis model for training; the tensor is put into a shared 2D convolutional neural network to extract feature maps. In order to ensure the convergence speed of the model and enable different hidden units in the network to learn different features, the 2D convolutional subnetwork can be pre-trained on the ImageNet dataset, and the stacked extracted feature maps are input into the 3D convolutional subnetwork. The 3D convolutional subnetwork is also pre-trained on Kinetics, and 3D convolution kernels are used in this cube, so that the features of time and space dimensions can be better integrated.

[0058] In a specific implementation, the anomaly analysis model may include multiple analysis channels, each analysis channel includes one of the above-mentioned deep learning models, and the number of analysis channels depends on the number of seats for the "old, weak, sick, disabled, and pregnant" passengers on the bus. Anomaly analysis can be performed in parallel on multiple segmented sub-images.

[0059] The physiological information analysis unit 205 is used to analyze the preprocessed physiological information signal, extract the BCG signal, respiratory information and body movement information therein through an algorithm and analyze them separately to obtain the human heart rate value and respiratory frequency and determine whether the passenger is in the seat; judge whether the analyzed physiological information is within the safety threshold. If it is within the safety threshold, the analysis result is normal; if it is not within the safety threshold, the analysis result is abnormal, wherein the safety threshold of the heartbeat is not more than 80 times / minute, and the safety threshold of the respiratory frequency is 12 times / minute-24 times / minute.

[0060] The abnormality determination unit 206 is used to determine whether the passenger has an abnormality caused by physical discomfort in combination with the analysis results of the video data analysis unit 204 and the physiological information data analysis unit 205. First, the video data and the physiological information are matched according to the acquisition time and the seat number. The matched video data and the physiological information are information of the same passenger. Secondly, if the analysis result of the video data analysis unit is that the passenger has an abnormality, and the analysis result of the physiological information data analysis unit is that the passenger has an abnormality, then it is determined that the passenger has an abnormality.

[0061] The alarm unit 207 is used to send an alarm to the monitoring personnel in the central monitoring room when the abnormality determination unit 206 determines that there is an abnormality in a passenger on the bus, and the monitoring personnel will make further confirmation based on the video and trigger the dispatch unit 208 after confirmation;

[0062] The dispatching unit 208 is used to dispatch an ambulance or an idle public bus to the location of the bus where the passenger is located, so as to provide medical treatment to the passenger and transfer other passengers.

[0063] An ambulance can be dispatched by calling 120 and informing the location of the bus where the passenger is, so that 120 can allocate the ambulance closest to the location for rescue; idle buses can be dispatched through the bus dispatch system within the bus company, and idle buses or the buses closest to the location can be dispatched to transport other passengers.

[0064] The bus terminal 300 includes: a video data acquisition and transmission unit 301 and a physiological information acquisition and transmission unit 302.

[0065] The video data acquisition and transmission unit 301 is used to acquire and transmit the video data of passengers in the bus compartment by using a camera;

[0066] The bus is equipped with a camera capable of capturing images of passengers in the bus compartment. The video data acquisition and transmission unit 301 collects video data of the passengers through the camera, and transmits the video data to the platform end using the wireless transmission channel between the bus and the central monitoring room.

[0067] The physiological information collection and transmission unit 302 is used to collect and transmit the physiological information of passengers on the bus seats using the physiological information monitoring pads arranged on each seat;

[0068] Physiological information monitoring pads (which can be seat cushions, backrests, or both) are placed on the seats for the elderly, the weak, the sick, the disabled, and pregnant women on the bus. Physiological information collection and transmission unit 302 collects the physiological information of passengers through the physiological information monitoring pads, and transmits the physiological information to the platform end through the wireless transmission channel between the bus and the central monitoring room. The physiological information collection and transmission unit here includes the physiological information monitoring pads placed on each seat.

[0069] Passengers in the "old, weak, sick, disabled, and pregnant" seats are generally elderly people, pregnant women, and patients. These passengers are prone to physical discomfort. In order to reduce the cost of use, physiological information monitoring pads are only set on these seats to collect only the physiological information of passengers in these seats. This arrangement can also achieve the technical effect of reducing the amount of data collected and analyzed.

[0070] Among them, the physiological information monitoring pad is provided with a physiological information collection device and an information transmission device for collecting physiological information inside its pad body, which can collect physiological information such as breathing rate and heartbeat of the person in contact with the physiological information monitoring pad, and transmit the collected information. The physiological information collection device includes a piezoelectric film sensor and a pressure sensor. The piezoelectric film sensor is a thin and soft sensor with high sensitivity. It can be used to detect weak change signals. Physiological information includes the human body's BCG (Ballistocardiogram) signal, respiratory information (respiratory rate, etc.), body movement information, etc.; the pressure sensor can be used to detect whether there is anyone on the seat. The physiological information monitoring pad is provided with a piezoelectric film sensor, which can non-invasively and non-sensingly monitor the user's heart rate, respiratory rate, HRV and other important physiological information. Passengers can monitor and record the customer's important physiological information in real time without wearing any additional instruments and equipment. The information transmission device can use the wireless transmission channel between the bus and the central monitoring room to transmit the physiological information collected by the physiological information collection device to the platform end by wireless communication, so that the platform end can use the obtained information to further analyze whether the passenger is feeling unwell, such as analyzing the user's stress state and emotions through HRV, and analyzing the user's cardiopulmonary state through heart rate, etc. The data sent by the information transmission device in each physiological information monitoring mat to the platform end is named after the bus and seat number where the physiological information monitoring mat is located.

[0071] The bus service intelligent management platform in the embodiment of the present invention monitors and manages the abnormalities caused by physical discomfort of passengers riding the bus in real time, adopts a monitoring method combining video and physiological signals, and timely and accurately detects the physical discomfort of passengers by analyzing the video images and physiological signals of passengers, ensuring timely and effective help when passengers have various conditions, and ensuring the safety of all personnel during the driving of the bus. At the same time, the bus service intelligent management platform in the embodiment of the present invention, in terms of hardware, only needs to arrange physiological signal monitoring pads on the "old, weak, sick, disabled, pregnant" special seats, and then use the existing cameras on the bus to implement it, without making major changes to the bus, and is easy to implement; in terms of software, it uses a neural network combined with 2D convolution and 3D convolution, which greatly improves the performance and efficiency of abnormal behavior recognition; at the same time, it adopts a combination of video image analysis and physiological signal analysis, as well as a passenger abnormality recognition judgment method confirmed by manual confirmation, which minimizes misjudgment and thus reduces the waste caused by resource scheduling after misjudgment.

[0072] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0074] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0075] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A public transportation business intelligent management platform, characterized in that: The public transportation business intelligent management platform mainly includes: a passenger monitoring and management system for monitoring abnormalities caused by physical discomfort of multiple passengers; the passenger monitoring and management system includes: a platform end and a bus end; wherein: The platform includes: A video information receiving unit for receiving video information transmitted from a bus terminal; A physiological information receiving unit for receiving physiological information transmitted by the bus terminal; A data preprocessing unit for preprocessing the video information obtained by the video information receiving unit and the physiological information obtained by the physiological information receiving unit; A video data analysis unit for receiving the sparsely processed data of the picture frames in the video stream transmitted by the data preprocessing unit, segmenting each passenger in the image using an image segmentation model for the sparsely processed data of the picture frames in the video stream, and then distinguishing the abnormal behavior and normal behavior of the passengers using an abnormal analysis model based on deep learning for each segmented sub-image, so as to obtain the distinction result of the normal behavior and abnormal behavior of the passengers when taking the bus; A physiological information analysis unit for analyzing the preprocessed physiological information signal, wherein the physiological information analysis unit extracts the BCG signal, respiratory information and body movement information therein and analyzes them respectively to obtain the human heart rate value and respiratory frequency and determine whether the passenger is in the seat; and judges whether the analyzed physiological information is within the safety threshold. If so, the analysis result is normal; if not, the analysis result is abnormal; An abnormality determination unit for determining whether the passenger has abnormality caused by physical discomfort based on the analysis results of the video data analysis unit and the physiological information data analysis unit; When the abnormality determination unit determines that a passenger on the bus is abnormal, an alarm is sent to the monitoring personnel in the central monitoring room, and the monitoring personnel further confirms based on the video and triggers the alarm unit of the dispatching unit after confirmation; A dispatching unit for dispatching an ambulance or an idle public bus to the location of the bus where the passenger is located, so as to provide medical treatment to the passenger and transfer other passengers; The bus terminal includes: a video data acquisition and transmission unit for acquiring and transmitting video data of passengers in a bus compartment using a camera, and, A physiological information collection and transmission unit is used to collect and transmit the physiological information of passengers on bus seats using physiological information monitoring pads arranged on each seat.

2. A public transportation business intelligent management platform according to claim 1, characterized in that: The image segmentation model adopts a mask-based regional convolutional neural network.

3. The public transportation business intelligent management platform according to claim 1, characterized in that: The anomaly analysis model is a deep learning model formed by combining the 2D convolutional subnetwork and the 3D convolutional subnetwork through feature splicing; The 2D convolution subnetwork is composed of four Ghost-Bottleneck blocks, which are stacked by two GhostModule modules and a depth-separable convolution module. The GhostModule is divided into main convolution operation and simple transformation operation. The main convolution generates a feature map, and then the simple transformation operation is used to transform the feature map generated by the main convolution. Finally, the two parts of the feature map are spliced ​​together and input to the next GhostModule. The attention mechanism of the SE module is added to the fourth Ghost Bottleneck block. The 3D convolution subnetwork adopts the structure of 3D-Shufflenetv2, which consists of 6 3D convolution modules. Each 3D convolution module is calculated by 1×1×1, 3×3×3 and 1×1×1 convolutions respectively, and finally merged together to the next 3D convolution module. Finally, the fully connected layer is used to classify abnormal and normal behaviors.

4. A public transportation business intelligent management platform according to claim 1, characterized in that: The physiological information monitoring pad is provided with a physiological information collection device and an information transmission device for collecting physiological information inside its pad body; The physiological information acquisition device includes a piezoelectric film sensor and a pressure sensor. The piezoelectric film sensor is used to detect weak change signals, based on which physiological signals can be obtained; the pressure sensor is used to detect whether there is anyone on the seat.

5. The public transportation business intelligent management platform according to claim 1, characterized in that: The physiological information includes respiratory rate and heart rate values.

6. The public transportation business intelligent management platform according to claim 1, characterized in that: The seats mentioned are reserved for the elderly, weak, sick, disabled and pregnant women.

7. A public transportation business intelligent management platform according to claim 6, characterized in that: For video image information, preprocessing includes cropping the target area and sparse processing of the picture frame. The target area is the area where the seats for the elderly, weak, sick, disabled, and pregnant women are located in the bus compartment; The target area is cropped for each image frame. The sparse processing refers to the two-step preprocessing of the image frame by sliding window. The first step is to select 8 seconds as a segment, take sparse sampling of 16 frames in a segment, and process these 16 frames into 224×224 size image frames. The second step is to convert the image frame into a tensor format acceptable to the model. For physiological information, preprocessing includes denoising and segmentation. Denoising is to remove power frequency interference and signal baseline to obtain original waveform data; Segmentation refers to distinguishing the physiological signals of different passengers in the seat. The physiological signals collected during the time period when a passenger is detected sitting on the seat and when a passenger leaves the seat are regarded as the physiological signals of one passenger based on the time node when the pressure sensor detects whether there is a passenger sitting on the seat.

8. The public transportation business intelligent management platform according to claim 1, characterized in that: The abnormality determination unit is specifically used for: Matching video data and physiological information according to the acquisition time and seat number, the matched video data and physiological information are information of the same passenger; If the analysis result of the video data analysis unit is that the passenger is abnormal, and the analysis result of the physiological information data analysis unit is that the passenger is abnormal, it is determined that the passenger is abnormal.

9. The public transportation business intelligent management platform according to claim 1, characterized in that: The safe threshold for heart rate is no more than 80 beats / minute, and the safe threshold for respiratory rate is 12-24 beats / minute.

Images