Intelligent barrier-free guiding system for square in front of rail transit station

Abstract

The invention relates to the technical field of rail transit, in particular to an intelligent barrier-free guiding system for a square in front of a rail transit station, which comprises an image collector and a self-service terminal which are deployed near a stand column of the square in front of the station, and an information processing device which is in communication connection with the image collector and the self-service terminal, the image collector is used for collecting an original image of the target area in real time; the information processing device is used for judging whether a disabled person exists in the target area or not according to the received original image; if yes, according to the original image and the first position information of the image collector, the recognition result of the disabled person is obtained, and according to the recognition result and the station entering and exiting route information of the station, a first control instruction is generated to control the self-service terminal to display and / or broadcast the station entering and / or exiting route corresponding to the disabled person in the original image through voice. The method has the beneficial effects that great convenience is brought to the rail travel of the disabled while the help intervention of workers is not needed.

Description

Technical Field

[0001] The present invention relates to the field of rail transit technology, and in particular to an intelligent barrier-free guidance system for a square in front of a rail transit station. Background Art

[0002] In recent years, urban construction has attached more and more importance to the construction of barrier-free environment, providing convenience for the disabled and the elderly to independently and safely pass through roads, enter and exit buildings and use their ancillary facilities, build public transportation tools, obtain, use and exchange information, and obtain social services. However, through research, it was found that the barrier-free system in the square in front of rail transit stations is still not perfect. There are many problems such as no slope, unreasonable slope, complex slope line, poor portability, discontinuous blind path, detour of blind path, and occupation of blind path space, which bring great inconvenience to the rail travel of people with motor dysfunction and visual impairment.

[0003] In order to solve the above problems, a rail transit operation and maintenance data processing system based on unmanned inspection (CN118229260A) is proposed, which includes a collection end, a database center and an operation and maintenance end; the database center sends operation and maintenance instructions to the operation and maintenance end after analyzing the data collected by the collection end; the collection end uses the collection device as a carrier, and the collection end includes a collection module and a first transmission unit; the collection module includes an image collection unit, and the image collection unit transmits the collected image data to the database center through the first transmission unit. That is, the image of passengers within the range of rail transit is collected through the collection module, and the disabled are identified by the processing module, thereby attracting the attention of the staff of the operation and maintenance end and providing timely assistance.

[0004] In summary, the current rail transit station facilities are highly dependent on the help of auxiliary personnel, and are incompatible with the behavioral habits of many disabled people and people with weak normal mobility, which affects their travel and brings great inconvenience to the travel of disabled people. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an intelligent barrier-free guidance system for the square in front of a rail transit station, which solves the technical problem that the facilities in front of the rail transit station are imperfect, resulting in inconvenience in travel for disabled people and people with weak normal mobility.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, an embodiment of the present invention provides a smart barrier-free guidance system for a square in front of a rail transit station, comprising: an image collector and a self-service terminal deployed near a column in the square in front of the station, and an information processing device communicatively connected to the image collector and the self-service terminal;

[0010] The image collector is used to collect the original image of the target area in real time;

[0011] The information processing device is used to determine whether there is a disabled person in the target area based on the original image collected in real time by the image collector;

[0012] If so, obtaining the identification result of the disabled person according to the original image and the preset first position information of the image collector, and generating a first control instruction according to the identification result and the preset station entry and exit route information to control the self-service terminal to display and / or voice broadcast the entry and / or exit route corresponding to the disabled person in the original image;

[0013] The identification result of the disabled person includes: the type of disability and the direction of movement of the disabled person.

[0014] Optionally, the information processing device determines whether there is a disabled person in the target area based on the original image collected in real time by the image collector, including:

[0015] Inputting the original image of the target area received in real time into the trained first disabled person recognition model, and obtaining a first judgment result of whether there is a disabled person in the target area output by the first disabled person recognition model;

[0016] The first disabled person recognition model includes a first disabled person image, a first normal pedestrian image, and a mixed image to train a YOLO model, thereby obtaining a first disabled person recognition model;

[0017] The first disabled person image includes at least one image of a pedestrian using an auxiliary device to move; the first normal pedestrian image includes at least one image of a normal pedestrian; the mixed image includes at least one image of a pedestrian using an auxiliary device to move and at least one image of a normal pedestrian;

[0018] The first disabled person recognition model is used to perform target detection on the received original image, identify pedestrians in the original image, and generate a corresponding bounding box, perform image segmentation based on the bounding box, extract each pedestrian from the original image, and generate at least one first sub-image;

[0019] Afterwards, feature extraction is performed on each first sub-image to generate a feature vector corresponding to each first sub-image, and based on the feature vector corresponding to each first sub-image, it is determined whether an auxiliary device exists in the corresponding first sub-image;

[0020] When an auxiliary device exists in any first sub-image, it is determined that a disabled person exists in the target area corresponding to the received original image.

[0021] Optionally, the information processing device obtains a recognition result of the disabled person according to the original image and the preset first position information of the image collector, and generates a first control instruction according to the recognition result and the preset station entry and exit route information, including:

[0022] Based on a preset image segmentation algorithm, the original image is segmented into at least one second sub-image; each of the second sub-images is an image containing only one pedestrian;

[0023] Inputting all the second sub-images into a pre-trained second disabled person recognition model to obtain a second judgment result of whether the pedestrian in the second sub-image is a disabled person, and marking the second sub-image corresponding to the second judgment result of the pedestrian being a disabled person;

[0024] The second disabled person recognition model comprises: a second disabled person recognition model obtained by training an SSD model with a second disabled person image and a second normal pedestrian image;

[0025] The second disabled person image includes an image of a pedestrian using an assistive device to move; the second normal pedestrian image includes an image of a normal pedestrian;

[0026] According to all the marked second sub-images and the preset first position information of the image collector, the recognition result of the disabled person in the second sub-image is obtained, and according to the recognition result and the preset station entry and exit route information, a first control instruction is generated and sent to the self-service terminal.

[0027] Optionally, the information processing device obtains a recognition result of the disabled person in the second sub-image according to all the annotated second sub-images and the preset first position information of the image collector, and generates a first control instruction according to the recognition result and the preset station entry and exit route information and sends it to the self-service terminal, including:

[0028] Input all the annotated second sub-images into a pre-trained auxiliary device detection model to obtain the auxiliary device type corresponding to each second sub-image;

[0029] Obtaining a recognition result of the disabled person according to the auxiliary device type corresponding to each second sub-image and the preset first position information of the image collector, and generating a first control instruction according to the recognition result and the preset station entry and exit route information and sending it to the self-service terminal;

[0030] The auxiliary device detection model includes a YOLO model trained with a preset sample training set to obtain an auxiliary device detection model;

[0031] The sample training set includes: at least one second disabled person image respectively labeled with a target detection frame and an auxiliary device type;

[0032] Types of assistive devices include canes, crutches, and wheelchairs.

[0033] Optionally, the information processing device inputs all the annotated second sub-images into a pre-trained auxiliary device detection model to obtain the auxiliary device type corresponding to each second sub-image, including:

[0034] Performing target detection on the received second sub-image, determining the auxiliary device target to be classified, and obtaining a quadruple of the target detection frame in each second sub-image in a pixel coordinate system;

[0035] The quaternion of the target detection frame includes the horizontal coordinate of the upper left corner of the target detection frame in the pixel coordinate system, the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, and the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection frame in the pixel coordinate system;

[0036] According to the quaternary group of the target detection frame and the preset formula 1, the feature parameters corresponding to the second sub-image are obtained; the formula 1 is:

[0037]

[0038] Wherein, Z is the feature parameter corresponding to the second sub-image, x1 is the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, y1 is the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, x2 is the horizontal coordinate of the upper left corner of the target detection frame in the pixel coordinate system, y2 is the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection frame in the pixel coordinate system, and α is a preset proportional coefficient;

[0039] According to the characteristic parameters corresponding to each second sub-image and a preset classification threshold, each second sub-image is classified, and the auxiliary device type corresponding to each second sub-image is obtained.

[0040] Optionally, the information processing device is further used for:

[0041] Based on the preset pedestrian key points, the pedestrian key points in each second sub-image are detected to obtain the confidence of each pedestrian key point on each second sub-image; the pedestrian key points include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left arm and right arm;

[0042] Based on the preset relationship between the parts of the human body structure and the confidence of the key points of each pedestrian in each second sub-image, the occlusion prediction score of each part of the disabled person is obtained;

[0043] Compare the occlusion prediction score of each part of the disabled person in each second sub-image with the preset part occlusion threshold, if the occlusion prediction score is less than the preset part occlusion threshold, the part is occluded;

[0044] According to the relationship between the parts of the human body structure, counting the occlusion ratio of pedestrian key points at the left and right boundaries of the disabled person in each second sub-image;

[0045] The direction of the disabled person in the second sub-image is determined according to the occlusion ratio of the pedestrian key points at the left boundary and the right boundary of the disabled person in the second sub-image.

[0046] Optionally, the information processing device obtains a recognition result of the disabled person according to the auxiliary device type corresponding to each second sub-image and the preset first position information of the image collector, and generates a first control instruction according to the recognition result and the preset station entry and exit route information and sends it to the self-service terminal, including:

[0047] Determine the movement direction of the disabled person in the second sub-image according to the orientation of the disabled person in the second sub-image and the first position information of the image collector;

[0048] Determine the disability type of the disabled person in the second sub-image according to the type of auxiliary equipment corresponding to the second sub-image; the disability type includes physical movement disorder and visual disorder;

[0049] According to the movement direction and disability type of the disabled person in the second sub-image, a first control instruction is generated and sent to the self-service terminal;

[0050] When the disability type of the disabled person in the second sub-image is physical movement disorder, the first control instruction includes an instruction for controlling the self-service terminal to display the entry and / or exit routes corresponding to the disabled person in the original image, and to display the locations of electric elevators, escalators and ramps;

[0051] When the disability type of the disabled person in the second sub-image is visual impairment, the first control instruction includes an instruction to control the self-service terminal to voice broadcast the entry and / or exit routes corresponding to the disabled person in the original image, as well as the location information of the blind path.

[0052] Optionally, the system further comprises a small mobile device communicatively connected to the information processing device;

[0053] The small mobile device is used to detect the second location information of the current small mobile device when receiving the first destination information input by the user, and send the second location information and the first destination information to the information processing device;

[0054] The information processing device is also used to generate a second control instruction based on the second location information and the first destination information, as well as the station's entry and exit route information, when receiving the second location information and the first destination information sent by the small mobile device, so as to control the small mobile device to display and / or voice broadcast the route corresponding to the first destination information input by the user.

[0055] Optionally, the information processing device is further used for:

[0056] Based on the received destination information and the pre-set urban traffic route map, the recommended bus connection route information is generated and sent to the small mobile device.

[0057] Optionally, the self-service terminal is also used for:

[0058] When receiving the second destination information input by the user, sending the second destination information to the information processing device;

[0059] The information processing device is also used to generate a third control instruction based on the second destination information and the preset self-service terminal location information when receiving the second destination information sent by the self-service terminal, so as to control the self-service terminal to display and / or voice broadcast the route corresponding to the second destination information input by the user.

[0060] (III) Beneficial effects

[0061] The beneficial effects of the present invention are as follows: the intelligent barrier-free guidance system for the square in front of a rail transit station of the present invention uses image processing technology to detect in real time whether there are disabled people, and provides corresponding display and / or voice broadcast assistance to the disabled people. Compared with the prior art, it does not require the help of staff and intervention, and brings great convenience to the rail travel of disabled people. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of the structure of an intelligent barrier-free guidance system for a square in front of a rail transit station provided by an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of the structure of a first disabled person identification model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0065] The square in front of a rail transit station: refers to the square in front of a subway or other rail transit station, which is usually an activity area for passengers to enter and exit the station and transfer transportation.

[0066] An intelligent barrier-free guidance system for a square in front of a rail transit station proposed in an embodiment of the present invention uses image processing technology to detect in real time whether there are disabled persons, and provides corresponding display and / or voice broadcast assistance to the disabled persons. Compared with the prior art, it does not require the intervention of staff, and brings great convenience to the rail travel of disabled persons.

[0067] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0068] Example 1

[0069] This embodiment provides a smart barrier-free guidance system for the square in front of a rail transit station. Figure 1 As shown, it includes: an image collector and a self-service terminal deployed near the pillars in the square in front of the station, and an information processing device that is communicatively connected with the image collector and the self-service terminal;

[0070] The image collector is used to collect the original image of the target area in real time;

[0071] An information processing device, used for judging whether there is a disabled person in the target area according to the original image collected in real time by the image collector;

[0072] If so, obtaining the identification result of the disabled person according to the original image and the preset first position information of the image collector, and generating a first control instruction according to the identification result and the preset station entry and exit route information to control the self-service terminal to display and / or voice broadcast the entry and / or exit route corresponding to the disabled person in the original image;

[0073] The identification result of the disabled person includes: the type of disability and the direction of movement of the disabled person.

[0074] The present embodiment provides an intelligent barrier-free guidance system for the square in front of a rail transit station. It uses image processing technology to detect in real time whether there are disabled persons, and provides corresponding display and / or voice broadcast assistance to the disabled persons. Compared with the prior art, it does not require the intervention of staff, and brings great convenience to the rail travel of disabled persons.

[0075] Example 2

[0076] This embodiment provides an intelligent barrier-free guidance system for a square in front of a rail transit station, comprising: at least one image collector and at least one self-service terminal, and an information processing device that is communicatively connected to all image collectors and all self-service terminals.

[0077] The image collectors and self-service terminals are evenly distributed near the pillars, corners, entrances and exits of the square in front of the rail transit station, so that pedestrians / passengers can use them at preset distances. The specific distance is determined based on the passenger flow surveyed in advance.

[0078] Normally, after the information processing device receives the original image captured by any image collector, it calls and sends control instructions to the self-service terminal at the appropriate location based on the preset pre-station equipment location distribution map and the location information of the image collector corresponding to the received original image.

[0079] Any image collector is used to collect the original image of its target area in real time;

[0080] An information processing device, configured to determine whether there is a disabled person in a target area based on the original image when receiving the original image sent by any image collector;

[0081] That is, the original image received in real time is input into the trained first disabled person recognition model to obtain a first judgment result of whether there is a disabled person in the target area output by the first disabled person recognition model;

[0082] The first disabled person recognition model includes a first disabled person image, a first normal pedestrian image, and a mixed image to train a YOLO model, thereby obtaining a first disabled person recognition model;

[0083] The first disabled person image includes at least one image of a pedestrian using an auxiliary device to move; the first normal pedestrian image includes at least one image of a normal pedestrian; the mixed image includes at least one image of a pedestrian using an auxiliary device to move and at least one image of a normal pedestrian;

[0084] The first disabled person recognition model is used to perform target detection on the received original image, identify pedestrians in the original image, and generate a corresponding bounding box, perform image segmentation based on the bounding box, extract each pedestrian from the original image, and generate at least one first sub-image;

[0085] Afterwards, feature extraction is performed on each first sub-image to generate a feature vector corresponding to each first sub-image, and based on the feature vector corresponding to each first sub-image, it is determined whether an auxiliary device exists in the corresponding first sub-image;

[0086] When an auxiliary device exists in any first sub-image, it is determined that a disabled person exists in the target area corresponding to the received original image.

[0087] The pre-trained first disabled person recognition model is used to perform preliminary recognition on the original image. At this time, it is only necessary to quickly identify whether there are disabled people in the original image to determine whether subsequent processing is required. On the basis of ensuring accuracy, processing resources are saved and processing speed is increased.

[0088] If there is a disabled person in the original image, the original image is segmented into at least one second sub-image based on a preset image segmentation algorithm; each second sub-image is an image of only one pedestrian; the image segmentation algorithm is used to segment the original image into areas with independent features, that is, to segment out all pedestrians and auxiliary equipment used by pedestrians in the original image.

[0089] Subsequently, all the second sub-images are input into a pre-trained second disabled person recognition model to obtain a second judgment result of whether the pedestrian in the second sub-image is a disabled person, and the second sub-image corresponding to the second judgment result of the disabled person is marked;

[0090] The second disabled person recognition model includes: a second disabled person recognition model obtained by training an SSD model with a second disabled person image and a second normal pedestrian image.

[0091] The SSD model is used to detect whether each pedestrian in the original image is a disabled person. Each second sub-image is re-analyzed at different scales / dimensions. While ensuring the accuracy of the analysis results, the high efficiency, simplicity and effectiveness of the SSD model are utilized to simplify the system structure and ensure the accuracy of the system operation.

[0092] The second disabled person image includes an image of a pedestrian using an assistive device to move; the second normal pedestrian image includes an image of a normal pedestrian;

[0093] Performing target detection on the received second sub-image, determining the auxiliary device target to be classified, and obtaining a quadruple of the target detection frame in each second sub-image in a pixel coordinate system;

[0094] The quaternion of the target detection frame includes the horizontal coordinate of the upper left corner of the target detection frame in the pixel coordinate system, the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, and the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection frame in the pixel coordinate system;

[0095] According to the quaternary group of the target detection frame and the preset formula 1, the feature parameters corresponding to the second sub-image are obtained; the formula 1 is:

[0096]

[0097] Wherein, Z is the feature parameter corresponding to the second sub-image, x1 is the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, y1 is the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, x2 is the horizontal coordinate of the upper left corner of the target detection frame in the pixel coordinate system, y2 is the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection frame in the pixel coordinate system, and α is a preset proportional coefficient;

[0098] According to the characteristic parameters corresponding to each second sub-image and a preset classification threshold, each second sub-image is classified, and the auxiliary device type corresponding to each second sub-image is obtained.

[0099] The auxiliary device detection model includes a YOLO model trained with a preset sample training set to obtain an auxiliary device detection model;

[0100] The sample training set includes: at least one second disabled person image respectively labeled with a target detection frame and an auxiliary device type;

[0101] Types of assistive devices include canes, crutches, and wheelchairs.

[0102] The auxiliary equipment in the second sub-image is scored using the quadruple of the target detection frame (i.e., feature parameters are constructed), and the scoring result is compared with the set classification threshold to output the corresponding classification result. This can accurately classify the auxiliary equipment to avoid misjudgment or missed judgment, and has high real-time performance. It is more suitable for the system proposed in this embodiment. Of course, it can be added and adjusted based on other auxiliary equipment not involved, so that the classification result is more in line with actual needs and the flexibility of the system is enhanced.

[0103] Subsequently, based on the preset pedestrian key points, the pedestrian key points in each second sub-image are detected to obtain the confidence of each pedestrian key point on each second sub-image; the pedestrian key points include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left arm and right arm;

[0104] Based on the preset relationship between the parts of the human body structure and the confidence of the key points of each pedestrian in each second sub-image, the occlusion prediction score of each part of the disabled person is obtained;

[0105] Compare the occlusion prediction score of each part of the disabled person in each second sub-image with the preset part occlusion threshold, if the occlusion prediction score is less than the preset part occlusion threshold, the part is occluded;

[0106] According to the relationship between the parts of the human body structure, counting the occlusion ratio of pedestrian key points at the left and right boundaries of the disabled person in each second sub-image;

[0107] The direction of the disabled person in the second sub-image is determined according to the occlusion ratio of the pedestrian key points at the left boundary and the right boundary of the disabled person in the second sub-image.

[0108] The human body structure is used to construct the pedestrian key points of the human body, and the orientation of the disabled person is analyzed based on the occlusion ratio to determine whether the disabled person is leaving or entering the station. The structure is simple and fast.

[0109] Determine the movement direction of the disabled person in the second sub-image according to the orientation of the disabled person in the second sub-image and the first position information of the image collector;

[0110] Determine the disability type of the disabled person in the second sub-image according to the type of auxiliary equipment corresponding to the second sub-image; the disability type includes physical movement disorder and visual disorder;

[0111] According to the movement direction and disability type of the disabled person in the second sub-image, a first control instruction is generated and sent to the self-service terminal;

[0112] When the disability type of the disabled person in the second sub-image is physical movement disorder, the first control instruction includes an instruction for controlling the self-service terminal to display the entry and / or exit routes corresponding to the disabled person in the original image, and to display the locations of electric elevators, escalators and ramps;

[0113] When the disability type of the disabled person in the second sub-image is visual impairment, the first control instruction includes an instruction to control the self-service terminal to voice broadcast the entry and / or exit routes corresponding to the disabled person in the original image, as well as the location information of the blind path.

[0114] When there are multiple disabled people in the original image, they can be displayed in a certain order, such as giving priority to entry and then exit, or giving priority to people with visual impairments, which can be set based on actual conditions.

[0115] The system further comprises:

[0116] The small mobile device is used to detect the second location information of the current small mobile device when receiving the first destination information input by the user, and send the second location information and the first destination information to the information processing device;

[0117] The information processing device is also used to generate a second control instruction based on the second location information and the first destination information, as well as the station's entry and exit route information, when receiving the second location information and the first destination information sent by the small mobile device, so as to control the small mobile device to display and / or voice broadcast the route corresponding to the first destination information input by the user.

[0118] A small mobile device that is easy to carry is set up, which can communicate with the information processing device via WiFi / Bluetooth. Of course, it can also be accessed and used by the disabled through the mobile phone / smartphone watch and other devices carried by the disabled after authentication. That is, access points / drop-off points with voice broadcast prompts are set up at the entrance of rail transit stations and vehicle alighting points for disabled people to choose to access.

[0119] The information processing device is also used for:

[0120] Based on the received destination information and the pre-set urban traffic route map, the recommended bus connection route information is generated and sent to the small mobile device.

[0121] The self-service terminal is further used to, when receiving the second destination information input by the user, send the second destination information to the information processing device;

[0122] The information processing device is also used to generate a third control instruction based on the second destination information and the preset self-service terminal location information when receiving the second destination information sent by the self-service terminal, so as to control the self-service terminal to display and / or voice broadcast the route corresponding to the second destination information input by the user.

[0123] Providing shuttle services for people with disabilities after leaving the station makes travel more convenient for them and enhances their convenience.

[0124] The information processing device of this embodiment identifies people with physical disabilities and visual disabilities through the original image, that is, identifies people with physical disabilities by identifying wheelchair users and fault users, and activates the display device of the self-service terminal. The display device displays the route through electric elevators, escalators, and ramps, and further has routes connecting buses and taxis, and broadcasts the locations and routes of electric elevators, escalators, and ramps to people with physical disabilities; identifies people with visual disabilities by identifying people with blind sticks, and activates the voice broadcasting device of the self-service terminal to broadcast the most convenient entry and exit routes for people with visual disabilities, whether the blind path is continuous, and other information to the visually impaired. If it is the exit direction, it further connects to bus, taxi and other route information. Furthermore, the display device can have a color-blind display mode for selection.

[0125] The present embodiment provides an intelligent barrier-free guidance system for the square in front of a rail transit station. It uses image processing technology to detect in real time whether there are disabled persons, and provides corresponding display and / or voice broadcast assistance to the disabled persons. Compared with the prior art, it does not require the intervention of staff, and brings great convenience to the rail travel of disabled persons.

[0126] Example 3

[0127] This embodiment provides a model training method. Taking the first disabled person recognition model as an example, the historical images (the first disabled person image, the first normal pedestrian image and the mixed image) collected by the image collector are used to construct a recognition training set.

[0128] A first disabled person recognition model is created. A YOLO-V10 model is usually selected. The YOLO-V10 model is initialized as a pre-trained first disabled person recognition model. The pre-trained first disabled person recognition model is trained using a recognition training set to obtain a trained first disabled person recognition model.

[0129] Before using the recognition training set to train the model, all images in the recognition training set need to be preprocessed to improve image quality and remove redundant information; that is, all images need to be sharpened to enhance the edges and details of the images;

[0130] All images are whitened to reduce redundant information and improve image contrast and clarity;

[0131] All images are processed with bilateral filtering to smooth the images and retain the edge and detail information of the images.

[0132] The first disability identification model, such as Figure 2 As shown, it usually includes the backbone network, the neck network and the head network;

[0133] The backbone network includes: a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a first downsampling module, a first C2fCIB module, a second downsampling module, a second C2fCIB module, an SPPF module and a PSA module connected in sequence;

[0134] The neck network includes: a first upsampling module, a first Concat module, a third C2fCIB module, a second upsampling module, a second Concat module, a third C2f module, a fourth Conv module, a third Concat module, a fourth C2fCIB module, a third downsampling module, a fourth Concat module and a fifth C2fCIB module connected in sequence;

[0135] The head network includes three Detct modules, and the three Detct modules are respectively connected to the fourth C2f module, the third C2fCIB module and the fourth C2fCIB module;

[0136] Wherein, the first C2fCIB module is connected to the second Concat module;

[0137] The second C2fCIB module is connected to the first Concat module;

[0138] The PSA module is connected to the first upsampling module and the fourth Concat module respectively;

[0139] The third C2fCIB module is connected to the third Concat module.

[0140] To improve accuracy, all upsampling modules in the neck network of the YOLO-V10 model can be replaced with DUpsampling modules.

[0141] Taking double upsampling as an example, the processing process of the DUpsampling module, for the input features, the DUpsampling module first uses a deep convolution with an output channel of 4 times and a kernel size to perform dimensionality increase calculations in the channel direction, and then expands the output after dimensionality increase in accordance with the channel dimension, and finally concatenates and combines the elements of the same empty position and different channels to form the final output result. The calculation process of the DUpsampling module (the whole process processing formula of the DUpsampling module) is as follows: Out DUp =R(DConv 1×1 (X));

[0142] Among them, X is the second original feature map received by the DUpsampling module, DConv 1×1 is a 1×1 depth-dimensional convolutional layer, R is an element rearrangement operation, Out DUpIt is the upsampled image output by the DUpsampling module.

[0143] Similarly, the training method of the auxiliary device detection model is similar to that of the first disabled person recognition model, and in terms of structure, only an auxiliary device feature parameter calculation module is added after the downsampling module of the backbone network, which is used to calculate the feature parameters of the auxiliary device, that is, according to the quadruple of the target detection frame and the preset formula 1, the feature parameters corresponding to the second sub-image are obtained; the formula 1 is:

[0144]

[0145] Among them, Z is the feature parameter corresponding to the second sub-image, x1 is the horizontal coordinate of the upper right corner of the target detection box in the pixel coordinate system, y1 is the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection box in the pixel coordinate system, x2 is the horizontal coordinate of the upper left corner of the target detection box in the pixel coordinate system, y2 is the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection box in the pixel coordinate system, and α is a pre-set proportional coefficient.

[0146] The training method of the second disabled person recognition model is similar to that of the first disabled person recognition model, except that the training set used is different. Its structure is as follows:

[0147] Image scaling module, which scales the input image to a 300*300*3 RGB image;

[0148] The backbone network is VGG16, which uses Conv5_3 (the third layer of the fifth convolution part) and previous structures of VGG-16. Through Conv4_3, the first feature prediction layer is obtained, and the output is a feature matrix of 38*38*512. Next, through the 3*3*1024 convolution layer, a feature matrix of 19*19*1024 is output (which can be regarded as the first fully connected layer of VGG16). Next, through the 1*1*1024 convolution layer, the second feature prediction layer is obtained, and the feature matrix of 19*19*1024 is output (which can be regarded as the second fully connected layer of VGG16). Next, through the 1*1*256 and 3*3*512 (the stride of this convolution layer is 2) convolution layers, the third feature prediction layer is obtained, and the feature matrix of 10*10*512 is output. Next, through the convolutional layers of 1*1*128 and 3*3*256 (the step size is 2), the fourth feature prediction layer is obtained, and the output is a 5*5*256 feature matrix. Next, through the convolutional layers of 1*1*128 and 3*3*256, the fifth feature prediction layer is obtained, and the output is a 3*3*256 feature matrix. Finally, through the convolutional layers of 1*1*128 and 3*3*256, the sixth feature prediction layer is obtained, and the output is a 1*1*256 feature matrix.

[0149] Output header, outputs the detection results.

[0150] Through the above process, we get 6 feature layers of SSD, which can predict objects of different sizes at these 6 prediction layers. For example, in the first feature prediction layer, it will detect relatively small objects (because the abstraction level is small at this time, more detailed information is retained). As the abstraction level deepens, the subsequent feature prediction layers will detect larger objects.

[0151] This embodiment provides a model training method to train a first disabled person recognition model, a second disabled person recognition model and an auxiliary equipment detection model with a certain degree of accuracy, providing a model basis for the implementation of a smart barrier-free guidance system for a rail transit station square provided in Embodiment 1 or Embodiment 2.

[0152] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0153] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0154] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0155] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0156] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An intelligent barrier-free guidance system for the square in front of a rail transit station, characterized in that: include: An image collector and a self-service terminal deployed near the pillars in the square in front of the station, and an information processing device connected to the image collector and the self-service terminal; The image collector is used to collect the original image of the target area in real time; The information processing device is used to determine whether there is a disabled person in the target area based on the original image collected in real time by the image collector; If so, obtaining the identification result of the disabled person according to the original image and the preset first position information of the image collector, and generating a first control instruction according to the identification result and the preset station entry and exit route information to control the self-service terminal to display and / or voice broadcast the entry and / or exit route corresponding to the disabled person in the original image; The identification result of the disabled person includes: the type of disability and the direction of movement of the disabled person.

2. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 1 is characterized in that: The information processing device determines whether there is a disabled person in the target area based on the original image collected in real time by the image collector, including: Inputting the original image of the target area received in real time into the trained first disabled person recognition model, and obtaining a first judgment result of whether there is a disabled person in the target area output by the first disabled person recognition model; The first disabled person recognition model includes a first disabled person image, a first normal pedestrian image, and a mixed image to train a YOLO model, thereby obtaining a first disabled person recognition model; The first disabled person image includes at least one image of a pedestrian using an auxiliary device to move; the first normal pedestrian image includes at least one image of a normal pedestrian; the mixed image includes at least one image of a pedestrian using an auxiliary device to move and at least one image of a normal pedestrian; The first disabled person recognition model is used to perform target detection on the received original image, identify pedestrians in the original image, and generate a corresponding bounding box, perform image segmentation based on the bounding box, extract each pedestrian from the original image, and generate at least one first sub-image; Afterwards, feature extraction is performed on each first sub-image to generate a feature vector corresponding to each first sub-image, and based on the feature vector corresponding to each first sub-image, it is determined whether an auxiliary device exists in the corresponding first sub-image; When an auxiliary device exists in any first sub-image, it is determined that a disabled person exists in the target area corresponding to the received original image.

3. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 1 is characterized in that: The information processing device obtains a recognition result of the disabled person according to the original image and the preset first position information of the image collector, and generates a first control instruction according to the recognition result and the preset station entry and exit route information, including: Based on a preset image segmentation algorithm, the original image is segmented into at least one second sub-image; each of the second sub-images is an image containing only one pedestrian; Inputting all the second sub-images into a pre-trained second disabled person recognition model to obtain a second judgment result of whether the pedestrian in the second sub-image is a disabled person, and marking the second sub-image corresponding to the second judgment result of the pedestrian being a disabled person; The second disabled person recognition model comprises: a second disabled person recognition model obtained by training an SSD model with a second disabled person image and a second normal pedestrian image; The second disabled person image includes an image of a pedestrian using an assistive device to move; the second normal pedestrian image includes an image of a normal pedestrian; According to all the marked second sub-images and the preset first position information of the image collector, the recognition result of the disabled person in the second sub-image is obtained, and according to the recognition result and the preset station entry and exit route information, a first control instruction is generated and sent to the self-service terminal.

4. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 3 is characterized in that: The information processing device obtains the recognition result of the disabled person in the second sub-image according to all the marked second sub-images and the preset first position information of the image collector, and generates a first control instruction according to the recognition result and the preset station entry and exit route information and sends it to the self-service terminal, including: Inputting all the annotated second sub-images into a pre-trained auxiliary device detection model to obtain the auxiliary device type corresponding to each second sub-image; Obtaining a recognition result of the disabled person according to the auxiliary device type corresponding to each second sub-image and the preset first position information of the image collector, and generating a first control instruction according to the recognition result and the preset station entry and exit route information and sending it to the self-service terminal; The auxiliary device detection model includes a YOLO model trained with a preset sample training set to obtain an auxiliary device detection model; The sample training set includes: at least one second disabled person image respectively labeled with a target detection frame and an auxiliary device type; Types of assistive devices include canes, crutches, and wheelchairs.

5. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 4 is characterized in that: The information processing device inputs all the annotated second sub-images into a pre-trained auxiliary device detection model to obtain the auxiliary device type corresponding to each second sub-image, including: Performing target detection on the received second sub-image, determining the auxiliary device target to be classified, and obtaining a quadruple of the target detection frame in each second sub-image in a pixel coordinate system; The quaternion of the target detection frame includes the horizontal coordinate of the upper left corner of the target detection frame in the pixel coordinate system, the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, and the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection frame in the pixel coordinate system; According to the quaternary group of the target detection frame and the preset formula 1, the feature parameters corresponding to the second sub-image are obtained; the formula 1 is: Wherein, Z is the feature parameter corresponding to the second sub-image, x1 is the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, y1 is the vertical coordinate of the horizontal coordinate of the upper right corner of the target detection frame in the pixel coordinate system, x2 is the horizontal coordinate of the upper left corner of the target detection frame in the pixel coordinate system, y2 is the vertical coordinate of the horizontal coordinate of the lower right corner of the target detection frame in the pixel coordinate system, and α is a preset proportional coefficient; According to the characteristic parameters corresponding to each second sub-image and a preset classification threshold, each second sub-image is classified, and the auxiliary device type corresponding to each second sub-image is obtained.

6. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 4 is characterized in that: The information processing device is also used for: Based on the preset pedestrian key points, the pedestrian key points in each second sub-image are detected to obtain the confidence of each pedestrian key point on each second sub-image; the pedestrian key points include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left arm and right arm; Based on the preset relationship between the parts of the human body structure and the confidence of the key points of each pedestrian in each second sub-image, the occlusion prediction score of each part of the disabled person is obtained; Compare the occlusion prediction score of each part of the disabled person in each second sub-image with the preset part occlusion threshold, if the occlusion prediction score is less than the preset part occlusion threshold, the part is occluded; According to the relationship between the parts of the human body structure, counting the occlusion ratio of pedestrian key points at the left and right boundaries of the disabled person in each second sub-image; The direction of the disabled person in the second sub-image is determined according to the occlusion ratio of the pedestrian key points at the left boundary and the right boundary of the disabled person in the second sub-image.

7. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 6 is characterized in that: The information processing device obtains the recognition result of the disabled person according to the auxiliary device type corresponding to each second sub-image and the preset first position information of the image collector, and generates a first control instruction and sends it to the self-service terminal according to the recognition result and the preset station entry and exit route information, including: Determine the movement direction of the disabled person in the second sub-image according to the orientation of the disabled person in the second sub-image and the first position information of the image collector; Determine the disability type of the disabled person in the second sub-image according to the type of auxiliary equipment corresponding to the second sub-image; the disability type includes physical movement disorder and visual disorder; According to the movement direction and disability type of the disabled person in the second sub-image, a first control instruction is generated and sent to the self-service terminal; When the disability type of the disabled person in the second sub-image is physical movement disorder, the first control instruction includes an instruction for controlling the self-service terminal to display the entry and / or exit routes corresponding to the disabled person in the original image, and to display the locations of electric elevators, escalators and ramps; When the disability type of the disabled person in the second sub-image is visual impairment, the first control instruction includes an instruction to control the self-service terminal to voice broadcast the entry and / or exit routes corresponding to the disabled person in the original image, as well as the location information of the blind path.

8. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 1 is characterized in that: The system also includes a small mobile device communicatively connected to the information processing device; The small mobile device is used to detect the second location information of the current small mobile device when receiving the first destination information input by the user, and send the second location information and the first destination information to the information processing device; The information processing device is also used to generate a second control instruction based on the second location information and the first destination information, as well as the station's entry and exit route information, when receiving the second location information and the first destination information sent by the small mobile device, so as to control the small mobile device to display and / or voice broadcast the route corresponding to the first destination information input by the user.

9. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 8 is characterized in that: The information processing device is also used for: Based on the received destination information and the pre-set urban traffic route map, the recommended bus connection route information is generated and sent to the small mobile device.

10. The intelligent barrier-free guidance system for the square in front of a rail transit station according to claim 1, characterized in that: The self-service terminal is also used for: When receiving the second destination information input by the user, sending the second destination information to the information processing device; The information processing device is also used to generate a third control instruction based on the second destination information and the preset self-service terminal location information when receiving the second destination information sent by the self-service terminal, so as to control the self-service terminal to display and / or voice broadcast the route corresponding to the second destination information input by the user.

Images