Intelligent boarding urging processing method, device and system based on multi-algorithm fusion

Abstract

The invention discloses an intelligent boarding urging processing method, device and system based on multi-algorithm fusion. The method comprises the steps that an airport terminal multi-algorithm fusion people searching service platform is built; the platform is connected with a security and protection system to obtain a video stream; communicating with a departure system to obtain information such as flights and passenger boarding; communicating with a security check information system to obtain passengers and over-check information thereof; and area broadcast can be played through connection with a broadcast system. Video analysis is carried out through various video analysis algorithms (facial features, human body features and spatial positioning) in the video analysis service platform, and the position and track of the passenger are judged; the flight information and the passenger information are combined, a video analysis algorithm result is fed back to an upper-layer application platform, and airport terminal passenger trajectory analysis is completed; through a broadcasting system, accurate and exclusive boarding prompting broadcasting is carried out in the area where the passengers arrive at night. According to the invention, passengers in an airport terminal are searched and positioned through multi-algorithm fusion, and an intelligent system capable of carrying out accurate and exclusive boarding promotion service is provided.

Description

Technical Field

[0001] The present invention relates to the field of passenger boarding service control, and in particular to an intelligent boarding urging processing method, device and system based on multi-algorithm fusion. Background Art

[0002] According to the actual historical operation of Chengdu Shuangliu International Airport, the flight passenger reduction rate in 2021 is 1.63%, and the average number of late passengers or non-boarding passengers in the terminal is about 200 per day. In the past, the urging boarding service for late passengers was mainly through gate broadcasts, general broadcasts or manual searches. There are the following problems: First, the coverage of gate broadcasts is small, and the success rate of urging boarding is low; second, for large-scale airports, the queuing time for general broadcasts during peak flight hours is long, and it is easy for passengers to board or the flight has been closed before the urging boarding broadcast is broadcast, and the waiting hall environment is noisy, and various broadcasts are frequently broadcast, so general broadcast searches often fail to attract the attention of late passengers; third, the staff cannot grasp the approximate location information of the passengers, and manual search is difficult.

[0003] To implement the smart travel in the Civil Aviation Administration's "Smart Civil Aviation Construction Roadmap"; adhering to the main strategic direction of Chengdu Shuangliu International Airport to create domestic boutique business routes, business jet routes and Hong Kong, Macao and Taiwan express routes, effectively attracting time-sensitive high-end passengers. Considering from the airport level, timely urging boarding and guidance services are provided for late passengers to improve the passenger experience; starting from the needs of airlines, assisting airline personnel to quickly locate passengers, predict flight closures, and reduce flight drop rates; starting from the needs of late passengers, hoping to enjoy fast and smooth boarding services to avoid missed flights. Summary of the invention

[0004] Therefore, in order to solve the above-mentioned shortcomings, a set of information systems that connect multiple information systems is designed, specifically providing an intelligent boarding urging processing method, equipment and system based on multi-algorithm fusion. The present invention uses multi-algorithm fusion to find and locate passengers in the terminal, and provides an intelligent system that can provide accurate and exclusive boarding urging services.

[0005] The present invention is implemented by constructing an intelligent boarding urging processing method based on multi-algorithm fusion, which is characterized by comprising the following process: Step 1: Build a terminal multi-algorithm fusion search service platform; Step 2: connect the above platform to the video surveillance system to obtain the terminal video surveillance video stream; Step 3, preprocess the video stream and convert the streaming media into images; Step 4, optimizing the image; Step 5: Perform recognition analysis to determine whether there is a person in the image and determine the person's location; Step 6: Feedback the video analysis algorithm results to the upper application platform to complete the passenger trajectory analysis in the terminal, generate the terminal passenger location, and form a passenger trajectory file.

[0006] Furthermore, in step 1, the terminal multi-algorithm fusion missing person service platform mainly completes the conversion of video unstructured data into structured data, extracts features at all levels based on video analysis technology, including macro features such as pedestrian detection coordinates and positions, and micro features such as pixels; establishes a real-time trajectory library and a historical trajectory library, and performs distributed storage on them; outputs the results and applies them to realize the retrieval of the trajectory and location of late passengers.

[0007] Furthermore, the operation of obtaining the video stream in step 2 includes: The video analysis service platform built in step 1 is connected to the video surveillance system through the streaming media server.

[0008] Further, the pre-processing in step 3 includes: The streaming media is converted into images through the image frame extraction algorithm; in order to process each image part separately, the image is segmented into background, face, and human body area outside the face through the image segmentation algorithm.

[0009] Furthermore, the image optimization operation in step 4 includes: Through the basic white balance algorithm and wide dynamic algorithm in digital image processing, the overall picture will not have obvious color cast, dark area and bright area mixed problems after processing; After the images are processed through the above steps, the super-resolution algorithm is used to improve the clarity of the images.

[0010] Furthermore, the algorithm identification analysis in step 5 includes: Through the face and body trajectory tracking algorithm, it is determined whether the passenger has passed through the location based on the face and body images detected in the above steps.

[0011] An intelligent boarding urging processing device based on multi-algorithm fusion; The main modules include: Preprocessing module: First, extract the video surveillance data into frames, then adjust the exposure, wide dynamic range, and white balance of the pictures to normal, and then complete the preparation work of human body detection and picture segmentation; Image optimization module: through the super-resolution function, it improves the image quality and face quality, which is convenient for algorithm analysis; Image recognition module: performs image recognition of facial features, clothing features, and behavior patterns; Application analysis module: Combines image recognition results with video surveillance system points to determine passenger trajectories and generate analysis reports.

[0012] An intelligent boarding urging processing system based on multi-algorithm fusion, comprising: The big data platform in the passenger smart service system based on face recognition is used as the data middle platform. The security information system is connected to the passenger inspection photo as a comparison photo, and the passenger inspection information is sent to the face boarding pad at the boarding gate. When the flight starts to urge boarding, the airline staff at the boarding gate will query the passengers who have not boarded on the boarding pad, and the departure system will push the information of the passengers who have not boarded to the pad. At the same time, the system is connected to the security video stream, and the video analysis is performed through a variety of video analysis algorithms (facial features, human features, spatial positioning) within the video analysis service platform to determine the location and trajectory of the passenger. The airline staff urges boarding with one click on the boarding gate pad, triggering the regional broadcast of the passenger's location to carry out exclusive urging boarding broadcast. Passengers can use the smart flight display nearby and quickly navigate to the boarding gate through face recognition; it can effectively improve the passenger boarding rate and reduce the flight passenger reduction rate; in addition, the airline staff at the boarding gate can query the inspection and location information of the passengers who have not boarded through the system, estimate the time they will arrive at the boarding gate, and assist the airline staff in predicting the closure of the flight, which is conducive to improving the flight regularity rate.

[0013] The present invention has the following advantages: 1. The system links multiple smart systems such as security information, departure, broadcasting, and security, turning a single smart system into a network of smart service projects.

[0014] 2. Breaking the original single and limited service of urging late passengers to board, airline staff at the boarding gate can obtain real-time information such as the image and location of passengers who have not boarded the plane within 5 seconds by performing simple operations on the boarding PAD, and urge them to board the plane through a regional broadcast containing the passenger's name, flight information and boarding gate information in the area where the passenger is located. At the same time, passengers can quickly navigate to the boarding gate through the nearby smart flight display face recognition. This method is efficient and convenient. Only one airline staff member at the boarding gate can complete a series of tasks such as boarding and searching for people, which saves manpower for airlines and can guide passengers to board more quickly, avoiding flight delays caused by too long time of manual search for passengers.

[0015] 3. Airline staff at the boarding gate can check whether passengers who have not boarded the plane have passed security checks on the facial boarding PAD, which can assist staff in predicting flight closures, which is conducive to improving flight regularity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flowchart of the terminal search and person-finding process based on multi-algorithm fusion; Figure 2This is a block diagram of the terminal missing person processing equipment module based on multi-algorithm fusion. DETAILED DESCRIPTION

[0017] The following will be combined with the attached Figure 1-Figure 2 The present invention is described in detail, and the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In view of the existing problems, the present invention designs an intelligent system which connects multiple information systems, searches and locates passengers in the terminal through the fusion of multiple algorithms, and provides precise and exclusive boarding urging services.

[0019] Using advanced visual recognition technology, high-definition cameras capture real-time images in the airport terminal, and use image processing and machine learning algorithms to automatically analyze the images. By identifying passengers' facial features, clothing types, colors, behavior patterns, etc., the location of passengers can be tracked and managed.

[0020] Use the security system cameras in the station building to find passengers, and connect the departure and security information systems to obtain passenger inspection and boarding information; the broadcasting system can efficiently and quickly urge passengers to board.

[0021] According to the MH / T7003-2017 document "Security and Protection Facilities for Civil Transport Airports", Shuangliu Airport has achieved full coverage of surveillance video. This solution focuses on the terminal waiting area, based on video data, combined with security inspection data, flight data, etc., with target recognition technology as the core, combined with re-identification technology, outputs the trajectory of late passengers and the location, camera ID and image information of the most recent time node, and sends it to the client or other business systems to narrow the scope of staff searching for people and actively remind late passengers.

[0022] Through the outbound passenger trajectory restoration and positioning linkage broadcasting system, late-arriving passengers can be found and accurately broadcast to urge them to board the plane, thus solving the problem of staff having difficulty finding people, increasing passenger boarding rates, and improving passengers' in-flight experience.

[0023] Embodiment 1, an intelligent boarding urging processing method based on multi-algorithm fusion; Figure 1 This is a flow chart of the terminal search method based on multi-algorithm fusion of the present invention. Figure 1 Provide a detailed description; Step S101, building a video analysis service platform; Build a video analysis service platform, which is the core of this system. It mainly completes the transformation of unstructured video data into structured data, and extracts features at all levels based on video analysis technology, including macro features such as pedestrian detection coordinates and micro features such as pixels. Establish a real-time trajectory library and a historical trajectory library, and store them in a distributed manner. Output the results and apply them to realize the retrieval of late-arriving passengers' trajectories and locations.

[0024] Step S102, obtaining a video stream; The video analysis service platform constructed in step S101 is connected to the video monitoring system through the streaming media server.

[0025] Step S103, preprocessing; The streaming media is converted into images through the image frame extraction algorithm. In order to process each part of the image separately, the image segmentation algorithm is used in this system to segment the image into background, face, and human body area outside the face.

[0026] Image segmentation algorithms include but are not limited to Mask R-CNN and FCN.

[0027] Step S104, image optimization; Through the basic white balance algorithm and wide dynamic algorithm in digital image processing, the overall picture will not have obvious color cast or unclear mixture of dark and bright areas after processing.

[0028] After the images are processed through the above steps, the system uses super-resolution algorithms to improve the clarity of the images.

[0029] Super-resolution algorithms include but are not limited to SRCNN and DRCN.

[0030] Step S105, algorithm identification and analysis; The system uses a human detection algorithm to determine whether there is a person in the image and determine the person's location.

[0031] Human body detection algorithms include but are not limited to HOG+AdaBoost and Faster-RCNN algorithms.

[0032] The system uses a face and body trajectory tracking algorithm to determine whether the passenger has passed through the location based on the face and body images detected in the above steps.

[0033] Step S106, application processing; The system forms a passenger trajectory file based on the terminal passenger location generated in step S105.

[0034] Embodiment 2, an intelligent boarding urging processing device based on multi-algorithm fusion; Figure 2It is a structural block diagram of the terminal building search processing equipment based on multi-algorithm fusion of the present invention; The main modules include: Preprocessing module: First, extract the video surveillance data into frames, then adjust the exposure, wide dynamic range, and white balance of the image to normal, and then complete the preparation work for human body detection and image segmentation.

[0035] Image optimization module: Through the super-resolution function, it improves the image quality and face quality to facilitate algorithm analysis.

[0036] Image recognition module: performs image recognition of facial features, clothing features, and behavioral patterns.

[0037] Application analysis module: Combines image recognition results with video surveillance system points to determine passenger trajectories and generate analysis reports.

[0038] Embodiment 3, the present invention also provides an intelligent boarding urging processing system based on multi-algorithm fusion, including using the big data platform in the passenger intelligent service system based on face recognition as the data middle platform, opening the security information system to use the passenger inspection photo as the comparison photo, and sending the passenger inspection information to the boarding gate pad. When the flight starts to urge boarding, the airline staff at the boarding gate inquires about the passengers who have not boarded on the boarding pad, and the departure system pushes the information of the passengers who have not boarded to the pad. At the same time, the system accesses the security video stream, and performs video analysis through multiple video analysis algorithms (facial features, human body features, spatial positioning) inside the video analysis service platform to determine the passenger's position and trajectory. The airline staff urges boarding with one click on the boarding pad, triggering the regional broadcast of the passenger's location to carry out exclusive urging boarding broadcast, and the passenger can use the smart flight display nearby to quickly navigate to the boarding gate through face recognition. It can effectively improve the passenger boarding rate and reduce the flight passenger reduction rate. In addition, airline staff at the boarding gate can use the system to query the inspection and location information of passengers who have not boarded the plane, estimate the time of their arrival at the boarding gate, and assist airline staff in predicting flight closures, which is conducive to improving flight regularity.

[0039] Advantages of the present invention: 1. The system links multiple smart systems such as security information, departure, broadcasting, and security, turning a single smart system into a network of smart service projects.

[0040] 2. Breaking the original single and limited service of urging late passengers to board, airline staff at the boarding gate can obtain real-time information such as the image and location of passengers who have not boarded the plane within 5 seconds by performing simple operations on the boarding PAD, and urge them to board the plane through accurate regional broadcasts containing the passenger's name, flight information and boarding gate information in the area where the passenger is located. At the same time, passengers can quickly navigate to the boarding gate through the nearby smart flight display face recognition. This method is efficient and convenient. Only one airline staff member at the boarding gate can complete a series of tasks such as boarding and searching for people, which saves manpower for airlines and can guide passengers to board more quickly, avoiding flight delays caused by too long time of manual search for passengers.

[0041] 3. Airline staff at the boarding gate can check whether passengers who have not boarded the plane have passed security checks on the facial boarding PAD, which can assist staff in predicting flight closures, which is conducive to improving flight regularity.

[0042] The workflow of the present invention is: 1. Obtain the video stream of the terminal video surveillance; 2. Build a terminal multi-algorithm fusion search service platform; 3. The above platform completes the interface connection with the video surveillance system to obtain the video stream; 4. Perform video analysis through various video analysis algorithms (facial features, human features, spatial positioning) within the video analysis service platform to determine the passenger's location and trajectory; 5. Feedback the video analysis algorithm results to the upper-level application platform to complete the passenger trajectory analysis in the terminal.

[0043] As described in the background technology, most airport terminals currently use the following methods to find late arriving passengers: (1) broadcasting in the vicinity of the boarding gate; (2) searching through general broadcasts; and (3) searching through passenger headshots obtained through security checks and manually watching videos.

[0044] The coverage of the boarding gate broadcast is small, and the success rate of urging boarding is low; for large airports, the waiting time for the general broadcast during peak flight hours is long, and it is easy for the passengers to board or the flight to be closed before the urging boarding broadcast is broadcast. In addition, the terminal environment is noisy and various broadcasts are broadcast frequently, so the general broadcast search often fails to attract the attention of late passengers; the quality of headshots is poor and can only provide facial feature information, and the staff cannot grasp the approximate location information of the passengers, and cannot expand the search range in a short time, which is inefficient.

[0045] Due to the different underlying technologies used, this system has some unique functions. For example: (1) Locate the last location of late-arriving passengers in seconds and display their location; (2) Supports displaying pictures of the passenger’s most recent location; (3) Display the passenger’s location trajectory within the last few minutes or within a custom time period; (4) Connect with the business system to automatically trigger the late-arriving passenger search function without manual input; (5) Independently judge whether to pass security check and only focus on passengers who have passed security check but have not boarded the plane; (6) Pay special attention to passengers in the concourse area where the boarding gate is located, and quickly locate the positions of late-arriving passengers nearby.

[0046] The innovative part of the present invention is that this application has designed a system that can determine the location and trajectory of passengers through the terminal video surveillance system, and accurately locate the location of late passengers based on the fusion of multiple algorithms such as human body, face, and spatial positioning, and can provide them with exclusive boarding urging and intelligent boarding guidance.

[0047] 1. Reduce flight drop rate by 80%, alleviating the pressure on gate staff to find passengers; 2. Increase the search speed of airline staff to 5 seconds; 3. Assist airline personnel to predict flight closures.

[0048] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent boarding urging processing method based on multi-algorithm fusion, characterized in that; The process includes the following: Step 1: Build a terminal multi-algorithm fusion search service platform; Step 2: connect the above platform to the video surveillance system to obtain the terminal video surveillance video stream; Step 3, preprocessing the video stream; Convert streaming media into images; Step 4, optimizing the image; Step 5: Perform recognition analysis to determine whether there is a person in the image and determine the person's location; Step 6: Feedback the video analysis algorithm results to the upper-level application platform to complete the terminal passenger trajectory analysis, generate the terminal passenger location, and form a passenger trajectory file.

2. According to claim 1, a method for intelligently urging boarding based on multi-algorithm fusion, characterized in that: In step 1, the terminal multi-algorithm fusion person-searching service platform mainly completes the transformation of video unstructured data into structured data, and extracts features at all levels based on video analysis technology, including macro features such as pedestrian detection coordinates and micro features such as pixels; Establish a real-time trajectory library and a historical trajectory library, and perform distributed storage on them; output the results to applications to achieve retrieval of late-arriving passengers’ trajectories and locations.

3. According to claim 1, the intelligent boarding urging processing method based on multi-algorithm fusion is characterized by: The operations for obtaining the video stream in step 2 include: The video analysis service platform built in step 1 is connected to the video surveillance system through the streaming media server.

4. According to claim 1, the intelligent boarding urging processing method based on multi-algorithm fusion is characterized in that ; The pre-processing in step 3 includes; The streaming media is converted into images through the image frame extraction algorithm; in order to process each image part separately, the image is segmented into background, face, and human body area outside the face through the image segmentation algorithm.

5. According to claim 1, the intelligent boarding urging processing method based on multi-algorithm fusion is characterized by: The image optimization operations in step 4 include: Through the basic white balance algorithm and wide dynamic algorithm in digital image processing, the overall picture will not have obvious color cast, dark area and bright area mixed problems after processing; After the images are processed through the above steps, the super-resolution algorithm is used to improve the clarity of the images.

6. According to claim 1, a method for intelligently urging boarding based on multi-algorithm fusion, The characteristic is that the algorithm identification analysis in step 5 includes: Through the face and body trajectory tracking algorithm, it is determined whether the passenger has passed through the location based on the face and body images detected in the above steps.

7. An intelligent boarding urging processing device based on multi-algorithm fusion, characterized in that ； The main modules include: Preprocessing module: First, extract the video surveillance data into frames, then adjust the exposure, wide dynamic range, and white balance of the picture to normal, and then complete the preparation work of human body detection and picture segmentation; Image optimization module: improves image quality and face quality through super-resolution function, which is convenient for algorithm analysis; Image recognition module: performs image recognition of facial features, clothing features, and behavior patterns; Application analysis module: Combines image recognition results with video surveillance system points to determine passenger trajectories and generate analysis reports.

8. An intelligent boarding urging processing system based on multi-algorithm fusion, characterized by: It includes using the big data platform in the passenger smart service system based on facial recognition as the data middle platform, connecting the security information system to use the passenger inspection photos as comparison photos, and sending the passenger inspection information to the facial boarding pad at the boarding gate. When the flight starts to urge boarding, the airline staff at the boarding gate will query the passengers who have not boarded on the boarding pad, and the departure system will push the information of the passengers who have not boarded to the pad. At the same time, the system is connected to the security video stream, and the video analysis is performed through a variety of video analysis algorithms (facial features, human features, spatial positioning) within the video analysis service platform to determine the passenger's location and trajectory. The airline staff urges boarding with one click on the boarding pad, triggering the regional broadcast of the passenger's location to carry out an exclusive urging boarding broadcast. Passengers can use the smart flight display nearby and quickly navigate to the boarding gate through facial recognition; it can effectively improve the passenger boarding rate and reduce the flight passenger reduction rate; in addition, the airline staff at the boarding gate can query the inspection and location information of the passengers who have not boarded through the system, estimate the time they will arrive at the boarding gate, and assist the airline staff in predicting the closure of the flight, which is conducive to improving the flight regularity rate.

Images