System and method for airport bus-linked checked baggage handling for baggage received through ai-based recognition and GPS coordinates with RFID-enhanced reliability

Abstract

Embodiments relate to an airport bus-linked checked baggage handling system and method for: calculating GPS data of an airport bus; capturing the face of a user boarding the airport bus to generate face image data of the user; analyzing the face image of the user to obtain identity data of the user; authenticating the user's intention to board an aircraft on the basis of GPS data of the user received from a terminal device of the identified user and the GPS data of the airport bus; requesting and acquiring, from the terminal device of the user, mobile aircraft boarding pass data of the user whose intention has been authenticated; and generating a baggage-linked code corresponding to the user's mobile aircraft boarding pass on the basis of baggage item data of the user boarding the airport bus. The airport bus-linked checked baggage handling system and method enable more reliable management of checked baggage received through AI-based recognition and GPS coordinates, by applying RFID technology to the checked baggage.

Description

Airport bus-linked checked baggage handling system and method for baggage received via AI-based recognition and GPS coordinates and reliability enhanced by RFID

[0001] Embodiments of the present application relate to an airport bus-linked checked baggage handling system and method in which the checked baggage handling procedure is initiated from the airport bus by linking the airport bus and the airport internal baggage inspection system based on authentication of a user's intent to board an aircraft using AI-based recognition and GPS coordinate-based location identification technologies.

[0002] In addition, embodiments of the present application relate to an airport bus-linked checked baggage handling system and method that enhances service reliability by applying RFID technology to checked baggage received via AI-based recognition and GPS coordinates.

[0003] Furthermore, embodiments of the present application relate to an airport bus-linked checked baggage handling system and method that provides separate transportation of the user and the checked baggage at the arrival airport for received checked baggage through AI-based recognition and GPS coordinates. This separate transportation process is reliably implemented through RFID technology.

[0004] When departing by aircraft, passengers must either carry their luggage directly onto the plane as carry-on baggage or load it into the cargo hold as checked baggage. While regulations regarding carry-on baggage vary by airline, generally, the weight allowed for carry-on baggage must fit within the cargo compartment with a total sum of its length, width, and height of less than 20 inches, while simultaneously satisfying weight limits. Therefore, passengers with a large amount of luggage must process their bags as checked baggage.

[0005] The checked baggage handling process involves the passenger arriving at the airport, purchasing a boarding pass or confirming their reservation at the designated airline counter, and checking in their baggage. This process can become significantly lengthy when there is a large queue at the check-in counter, making it one of the main sources of inconvenience for passengers.

[0006] Due to advancements in mobile technology, pre-check-in (or mobile check-in), which allows passengers to complete the boarding pass issuance process in advance before arriving at the airport without the need to obtain a boarding pass at the check-in counter, is being widely used.

[0007] However, there is a problem in that even passengers who have pre-checked in must visit the check-in counter to check in their luggage, so they cannot be free from waiting at the check-in counter.

[0008] To ensure that the benefits of pre-check-in are not diminished, technology is required that eliminates the need to visit designated check-in points within the airport, such as check-in counters.

[0009] Based on the discussion described above, the embodiments of the present application aim to provide an airport bus checked baggage linkage system and method using AI-based recognition and GPS coordinates, configured to link checked baggage loaded by a passenger onto an airport bus to an airport internal baggage management system at the time of boarding by utilizing aircraft boarding intent authentication technology that utilizes AI-based recognition and GPS coordinate-based location identification technology.

[0010] Furthermore, based on the discussion described above, the embodiments of the present application aim to provide an airport bus-linked checked baggage handling system and method that provides separate transportation at the arrival airport for baggage received via AI-based recognition and GPS coordinates and whose reliability is enhanced by RFID.

[0011] A checked baggage handling system using AI-based recognition and GPS coordinates according to one aspect of the present application comprises: a user terminal device that acquires boarding pass data for an airport bus to be used by a user to arrive at an airport to board an aircraft, generates a baggage bag image taken of the user’s baggage bag to be loaded onto the aircraft and a bag interior image showing the interior of the user’s baggage bag, analyzes the images to calculate the user’s baggage data, and transmits it to a server—the baggage data includes baggage data and baggage item data—; An airport bus terminal device that verifies the identity of the user by analyzing a facial image of the user boarding the airport bus, verifies the airport bus boarding pass of the user boarding the airport bus, authenticates the user's intention to board an aircraft based on the user's GPS data received from the verified user's terminal device and the airport bus's GPS data, requests and obtains the user's mobile aircraft boarding pass data from the user's terminal device or server after the user's intention to board the aircraft has been authenticated, and generates a baggage linkage code corresponding to the user's mobile aircraft boarding pass based on the user's baggage data - the user's mobile aircraft boarding pass data is stored in advance on a server or on the user's terminal device in association with the user's account information through a pre-check-in process, and the baggage linkage code is a code different from the airline baggage identification code -;The system may include a server configured to generate and manage baggage tag information, including a baggage linkage code and an airline's baggage identification code, in a digital form based on a baggage linkage code transmitted from a terminal device of the airport bus, and to receive a security inspection result including a second baggage bag image captured by the airport's baggage inspection system and a reading result data obtained from inspection source data for the baggage bag, wherein the reading result data includes an identification result for at least one item contained in the user's baggage bag, and to associate the reading result data for the baggage bag with the user's baggage linkage code based on the matching result between the second baggage bag image of the baggage inspection system and the user's baggage bag image, and the matching result between the reading result data of the baggage inspection system and the user's baggage item data, and to transmit a baggage check-in notification message to the user's terminal device based on an inspection result message for the user's baggage received from the baggage inspection system.

[0012] In one embodiment, the terminal device of the airport bus determines whether the airport bus has entered a pre-set designated section based on the GPS data of the airport bus, wherein the designated section is a certain section including an airport stop among the entire operating section of the airport bus, and when the airport bus has entered the pre-set designated section, determines whether the GPS location of the user and the GPS location of the airport bus match at at least one point on the designated section, and if it is determined that the GPS location of the user and the GPS location of the airport bus match at at least one point on the designated section, the terminal device may be configured to acknowledge the user's intention to board an aircraft.

[0013] In one embodiment, the user's terminal device may be configured to apply the luggage bag image to a first deep learning-based recognition model to recognize the user's luggage bag in order to calculate the user's luggage item data, search for the weight data of the user's luggage bag through the internet or a server based on the recognition result of the luggage bag, apply the user's bag interior image to a deep learning-based segmentation model to segment the item area where luggage appears in the bag interior image by class of luggage, extract the segmented item area as a sub-image, apply the sub-image of the extracted item area to a second deep learning-based recognition model to recognize the item appearing in the bag interior image, search for the weight data of the item inside the user's luggage bag through the internet or a server based on the recognition result of the item, and calculate the user's luggage weight information based on the search result of the weight of the recognized luggage bag and the search result of the weight of the recognized item.

[0014] In one embodiment, the user's terminal device may be further configured to request input of information regarding an item inside the pouch included in the pouch when the class of the recognized item corresponds to a pouch, and to search for weight data for the pouch based on the input information regarding the item inside the pouch and the size data of the pouch calculated from the segmentation result of the item area where the pouch appears.

[0015] In one embodiment, the deep learning-based segmentation model is trained using a first training data set, wherein the first training data set consists of a plurality of training samples, each training sample consists of training data and label data, and the training data includes at least one training image, which is a sample image of an incomplete shape representing a training sample item having an incomplete shape, and a sample image of a complete shape representing an item having a complete shape as an item appearing in the sample image of the incomplete shape, and the label data may represent the actual value of the item type of the luggage appearing in the training image.

[0016] In one embodiment, the second deep learning-based recognition model includes an input layer that inputs the sub-image as an input image, an encoder including one or more encoding blocks, a decoder including one or more decoding blocks, and a classification layer that recognizes and outputs an item appearing in the sub-image based on an encoding vector output from the decoder. The second deep learning-based recognition model is configured to divide the input sub-image into a plurality of patches—each patch having a fixed size—convert each patch into a patch embedding vector, combine a position embedding value representing the position of each patch with the patch embedding vector of the corresponding patch, input the combined vector into the encoder for processing to produce an encoding vector, recognize an item appearing in the sub-image based on the encoding vector, and output a recognition result. The encoder outputs a class token as the encoding vector through a multi-head attention mechanism and can learn the relationship between the plurality of patches through the multi-head attention mechanism.

[0017] In one embodiment, the second deep learning-based recognition model may be a foundation model pre-trained with a specific dataset that is fine-tuned to recognize incomplete or complete shaped items contained within a luggage bag.

[0018] In one embodiment, the user's terminal device is configured to calculate size information of an item appearing in an item area based on size information of the luggage bag and a segmentation result, and to apply the corresponding sub-image showing the luggage and the size information of the item to the second deep learning-based recognition model, and the second deep learning-based recognition model may be configured to convert the normalized size information of the item into a size information tensor, generate tensor data of the luggage based on a feature map extracted from the sub-image and the size information tensor, input the generated tensor data of the luggage into a fully connected layer as an intermediate operation result, and recognize an item appearing in a sub-image having an incomplete shape by considering the features of the luggage with an incomplete shape through the fully connected layer.

[0019] In one embodiment, the server may be configured to transmit a first notification message, including a baggage verification location and a baggage linkage code, to the user's terminal device based on a baggage linkage code within the baggage linkage code, upon receiving a rejection message from the baggage inspection system indicating that restricted checked baggage was detected during the baggage inspection, and to transmit a second notification message, including a baggage management number issued by the baggage inspection system, to the user's terminal device upon receiving a pass message from the baggage inspection system indicating that restricted checked baggage was not detected during the baggage inspection.

[0020] In one embodiment, the user's terminal device may be further configured to, when calculating the baggage data, check whether the user's baggage data satisfies the pre-set airline's checked baggage weight limit, check whether the user's baggage satisfies the pre-set checked baggage restriction guide based on the user's baggage data, provide a first warning message including checked baggage weight information if the user's baggage falls within the checked baggage weight limit, provide a second warning message including the detection result of the checked baggage restriction item if the recognized baggage items include a checked baggage restriction item, provide a pass message to the user if the baggage does not fall within the checked baggage weight limit and no checked baggage restriction item is identified, and update the user's baggage data according to a user input indicating the removal of baggage items entered after providing the first warning message or the second warning message.

[0021] In one embodiment, if the terminal device of the airport bus fails to verify identity as a result of analyzing the user's face image, the terminal device of the airport bus outputs the fact of the failure to verify identity through an output device, and the user's terminal device captures a second face image through a UI screen capable of capturing the user's real-time face at the time of boarding, inputs the second face image into a pre-trained identity verification model to obtain the user's identity information, and is configured to transmit the obtained user's identity information to the server or the terminal device of the airport bus.

[0022] In one embodiment, the terminal device of the airport bus may be configured to output a message indicating that a boarding pass exists through an output device when there is a mobile airport bus boarding pass associated with the user's account information, and to proceed with payment based on the user's payment method pre-registered in the system and output a message indicating that the payment has been completed when there is no mobile airport bus boarding pass associated with the user's account information.

[0023] In one embodiment, the checked baggage handling system further includes an employee terminal device configured to receive baggage tag information from the server and print it as a label sticker, and the employee terminal device may be configured to communicate with the server while moving with the employee in charge of the process from when the airport bus arrives at the airport to the baggage inspection system inside the airport during the airport bus-linked checked baggage handling process.

[0024] In one embodiment, the server may be configured to transmit a third notification message, including the user's baggage linkage code and baggage verification location, to the user's terminal device when the baggage bag shown in the second baggage bag image of the baggage inspection system and the baggage bag shown in the user's baggage bag image do not match each other.

[0025] In one embodiment, the server may be configured to, upon obtaining the user's baggage linkage code, associate the baggage linkage code with the user's account information and aircraft boarding pass information, and transmit a linkage service reception message including the user's baggage tag information to the user's terminal device.

[0026] In one embodiment, the checked baggage handling system further includes an employee terminal device connected to an RFID reader module to store the baggage tag information in an initialized RFID tag, and the server may be configured to receive the user's baggage tag information transmitted to the RFID tag from the RFID reader module and the UID of the RFID tag, associate them with each other, and update the usage history of the user having the baggage tag information associated with the UID of the RFID tag.

[0027] In one embodiment, the user's terminal device is configured to obtain information on tourist destinations the user will visit after landing the aircraft and information on accommodations the user will stay at, and the server may be configured to generate a user group list that groups users of the same destination based on the user's tourist destinations and accommodations information, transmit it to a specific transport vehicle staff terminal device in the arrival area and a specific user transport vehicle staff terminal device, and transmit a service schedule notification message to the user's terminal device that includes information on the boarding location of the specific user transport vehicle the user will board and information on the destination of the specific transport vehicle where the user's luggage will be transported.

[0028] In one embodiment, the employee terminal device of the specific transport vehicle is configured to read tag information stored in an RFID tag attached to a luggage bag through the RFID reader module, perform a first verification by comparing the read tag information with the luggage tag information of the user obtained from the user group list, and perform a second verification by comparing the UID of the RFID tag with the UID stored in the server, and the server may be configured to verify the validity of the TOTP generated from the RFID tag at the request of the employee terminal device.

[0029] In one embodiment, the server may be configured to calculate the matching probability of the luggage bag appearing in each bag image by comparing and analyzing the second luggage bag image of the luggage inspection system and the user's luggage bag image, acquire the user's luggage item data based on the luggage tag information appearing in the second luggage bag image, and calculate the matching probability by comparing and analyzing the reading result data of the luggage inspection system and the acquired user's luggage item data.

[0030] In one embodiment, the RFID tag may be configured to encrypt the user's baggage tag information received through the RFID reader module and store the encrypted user's baggage tag information in memory.

[0031] In one embodiment, the employee terminal device of the specific user transport vehicle is configured to perform boarding verification by comparing the user group list with the list of identity information of the user who boarded the specific user transport vehicle, and to transmit the boarding verification result to the server, and the server may be configured to update the user's usage history based on the boarding verification result.

[0032] In one embodiment, the destination of the transport vehicle transporting the user's luggage bag and the destination of the user transport vehicle transporting the user may be different places.

[0033] A computer-readable recording medium according to another aspect of the present application may record a program for performing a method of operation of a baggage handling system according to the embodiments described above.

[0034] An airport bus-linked checked baggage handling system according to various embodiments of the present application is configured to automatically proceed with the checked baggage handling process when baggage is loaded onto the airport bus at the time of boarding by linking the airport bus with an airport internal baggage inspection system using AI-based recognition and GPS coordinates. Unlike conventional checked baggage handling processes where a user must retrieve their baggage from the airport bus and drag it to the airline's check-in desk, the airport bus-linked checked baggage handling system eliminates the need for the user to retrieve their baggage from the airport bus or visit the airline's check-in desk. In other words, the checked baggage handling process begins with loading baggage onto the airport bus.

[0035] As a result, passengers scheduled to board an aircraft may be relieved of the burden of having to visit designated locations within the airport, such as check-in counters. Passengers can minimize the burden of the time required to complete check-in and baggage procedures before boarding time. Ultimately, passengers can use a fast and convenient aircraft, thereby improving the convenience of departure for users.

[0036] Furthermore, the effect of eliminating the need to visit the check-in desk in this airport bus-linked checked baggage handling system can provide additional benefits such as energy savings and enhanced transportation competitiveness for airport buses, as it increases the likelihood that passengers intending to board an aircraft will use airport buses instead of personal vehicles when approaching the airport.

[0037] In addition, the effect of eliminating the need to visit the check-in desk in this airport bus-linked baggage handling system is that the conventional check-in desk space, including the passenger waiting area, can be replaced with other interior spaces, ultimately allowing for more efficient utilization of the interior space for the boarding of aircraft users.

[0038] Furthermore, through AI-based recognition and GPS coordinates, the user's luggage bags can be independently transported to the hotel and to the tourist destination for the received baggage.

[0039] To more clearly explain the technical solution of the embodiments of the present invention or the prior art, the drawings necessary for the description of the embodiments are briefly introduced below. It should be understood that the drawings below are for the purpose of explaining the embodiments of this specification only and are not for the purpose of limitation. Additionally, for clarity of explanation, some elements in the drawings below may be depicted with various modifications, such as exaggeration or omission.

[0040] FIG. 1 illustrates a network environment of an airport bus-linked checked baggage handling system that provides separate transport for received baggage at the arrival airport through AI-based recognition and GPS coordinates, according to one aspect of the present application.

[0041] FIG. 2 is a schematic diagram of a neural network according to various embodiments of the present application.

[0042] FIG. 3 is a configuration diagram of a terminal device of an airport bus according to various embodiments of the present application.

[0043] FIG. 4 is a configuration diagram of an RFID tag according to various embodiments of the present application.

[0044] FIG. 5 is a flowchart of an airport bus-linked checked baggage handling method according to various embodiments of the present application.

[0045] FIG. 6 is a detailed flowchart of a process for calculating user baggage data according to various embodiments of the present application.

[0046] FIG. 7 illustrates an image of the inside of a luggage bag taken according to various embodiments of the present application.

[0047] FIG. 8 illustrates the result of dividing clothing items contained in a luggage bag according to various embodiments of the present application.

[0048] FIG. 9 is a model network structure diagram of a second deep learning-based recognition model according to various embodiments of the present application.

[0049] FIG. 10 is a flowchart of the learning process of a second deep learning-based recognition model including a first encoder and a decoder, according to specific embodiments of the present application.

[0050] FIG. 11 is a drawing illustrating restricted baggage according to various embodiments of the present application.

[0051] FIG. 12 is a schematic diagram of a baggage linkage service reception message according to various embodiments of the present application.

[0052] FIG. 13 is a detailed flowchart of the process of generating baggage tag information according to various embodiments of the present application.

[0053] FIG. 14 is a detailed flowchart of a process for verifying baggage tag information stored in an RFID tag according to various embodiments of the present application.

[0054] FIG. 15 is a detailed flowchart of a process for transmitting a baggage check-in notification message according to various embodiments of the present application.

[0055] FIG. 16 is a detailed flowchart of the process of generating a list of user groups according to various embodiments of the present application and transmitting it to the employee terminal device (1210) of a tour bus and the employee (1200) of a transport vehicle, respectively.

[0056] FIG. 17 is a detailed flowchart of the process of transporting luggage bags of a user group in a transport vehicle according to various embodiments of the present application.

[0057] FIG. 18 is a detailed flowchart of the process of handling members of a user group boarding a tour bus according to various embodiments of the present application.

[0058]

[0059] Hereinafter, embodiments of the present invention will be examined in detail with reference to the drawings.

[0060] However, this is not intended to limit the present disclosure to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives to the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0061] In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., components such as numbers, functions, actions, steps, parts, elements, and / or parts), and do not exclude the presence or addition of additional features.

[0062] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0063] Expressions such as "first," "second," "first," or "second" used in various embodiments may modify various components regardless of order and / or importance and do not limit said components. Such expressions may be used to distinguish one component from another. For example, the first component and the second component may represent different components regardless of order or importance.

[0064] Examples of configurations of singular expressions used in this specification also include examples of configurations of plural expressions, unless the phrases associated with the singular expressions clearly indicate a meaning contrary to this.

[0065] As used herein, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only that which is “specifically designed to” in hardware. Instead, in some situations, the expression “device configured to” may mean that the device is “capable of” in conjunction with other devices or components. For example, the phrase “processor configured to perform A, B, and C” may mean a dedicated processor for performing the said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in a memory device.

[0066] Terms used in this invention, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this invention. Terms used in this invention that are defined in general dictionaries may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not interpreted in an ideal or overly formal sense unless explicitly defined in this invention. In some cases, even terms defined in this invention may not be interpreted to exclude embodiments of this invention.

[0067]

[0068] An airport bus-linked checked baggage handling system using AI-based recognition and GPS coordinates (1, hereinafter referred to as the airport bus-linked checked baggage handling system) is configured to link an airport bus and a baggage inspection system by transmitting the user's baggage, which is transported by an airport bus from the user's boarding point to the airport bus stop, to a baggage inspection system installed at the airport. In this specification, baggage refers to a combination of baggage bags and baggage items.

[0069] FIG. 1 illustrates a network environment of an airport bus-linked checked baggage handling system that provides separate transport at the arrival airport for baggage received via AI-based recognition and GPS coordinates and whose reliability is enhanced by RFID, according to one aspect of the present application.

[0070] FIG. 1 illustrates a network environment of an airport bus-linked checked baggage handling system that provides separate transport at an arrival airport for baggage received via AI-based recognition and GPS coordinates, according to one aspect of the present application. FIG. 1a illustrates a network environment of the boarding stage portion of the airport bus-linked checked baggage handling system. FIG. 1b illustrates a network environment of the arrival stage portion of the airport bus-linked checked baggage handling system.

[0071] Referring to FIG. 1, the airport bus-linked checked baggage handling system (1) may include a terminal device (100) for a user who must arrive at an airport to board an aircraft, a terminal device (200) for an airport bus that the user will board to the airport, a server (300) that provides a linked checked baggage handling service, a staff terminal device (600) at the boarding airport, and a baggage inspection system (800) installed at the airport to perform the task of loading checked baggage onto an aircraft.

[0072] The airport bus-linked checked baggage handling system (1) according to the embodiments may be entirely hardware or have aspects that are partially hardware and partially software. For example, the system may collectively refer to hardware equipped with data processing capabilities and operating software for driving the same. In this specification, terms such as "unit," "system," and "device" are intended to refer to a combination of hardware and software driven by said hardware. For example, the hardware may be a data processing device including a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), or other processors. Additionally, the software may refer to a running process, an object, an executable file, a thread of execution, a program, etc.

[0073] The user's terminal device (100), the airport bus's terminal device (200), the server (300), and the baggage inspection system (800) can be connected to each other through an electrical communication network.

[0074] The telecommunications network provides a wired / wireless telecommunications path through which the components (100, 200, 300, 600) of the airport bus linked baggage handling system (1) can transmit and receive data to and from each other.

[0075] Telecommunication networks are not limited to communication methods based on specific communication protocols, and appropriate communication methods may be used depending on the implementation. For example, if configured as a system based on the Internet Protocol (IP), the telecommunication network may be implemented as a wired and / or wireless internet network. Or, if different devices (100, 200, or 300) are implemented as mobile communication terminals, the telecommunication network may be implemented as a wireless network such as a cellular network or a wireless local area network (WLAN) network.

[0076]

[0077] In the above airport bus-linked checked baggage handling system (1), the user's checked baggage is accepted as service-eligible baggage through AI-based recognition and GPS coordinate-based authentication of intent to board an aircraft. This acceptance process is implemented through the interaction between the user's terminal device (100), the airport bus terminal device (2000), and the server (300) below.

[0078]

[0079] The user's terminal device (100), the airport bus terminal device (200), and the employee terminal device (600) may be client terminal devices that communicate with the server (300).

[0080] The user's terminal device (100), the airport bus terminal device (200), and the employee terminal device (600) may be a computing system capable of performing appropriate functions implemented or supported by the device (100 or 200), including hardware, software or embedded logic components or a combination of two or more of these components.

[0081] The above user terminal device (100) is a device including a processor, memory, communication unit, input device, and output device, and may be implemented in the form of a computer system such as, for example, a desktop computer, a laptop computer, a netbook, a tablet computer, an e-book reader, a GPS device, a camera, a personal information terminal (PDA), a portable electronic device, a cellular phone, a smartphone, other computing device, other mobile device, other wearable device, other suitable electronic device, or any suitable combination thereof.

[0082] The user's terminal device (100) may directly acquire mobile boarding pass data through the boarding pass purchase process, or indirectly acquire mobile boarding pass data by performing and transmitting the boarding pass purchase process from another external device. The boarding pass purchase process may be a self-contained boarding pass purchase process included in a dedicated application provided by the airport bus-linked checked baggage handling system (1). Alternatively, the boarding pass purchase process may be provided by accessing a website selling boarding passes (e.g., an airline, a travel agency, or a boarding pass sales platform), or through another application selling boarding passes installed on the user's terminal device (100) separately from the dedicated application.

[0083] Additionally, the user's terminal device (100) may obtain mobile airport bus boarding pass data to be used by the user to arrive at the airport to board an aircraft through a dedicated application provided by the airport bus linked checked baggage handling system (1), or indirectly obtain mobile airport bus boarding pass data by performing and transmitting the airport bus boarding pass purchase process from another external device. To this end, the user's terminal device (100) may access a website selling airport bus boarding passes (e.g., an airport bus homepage, etc.) or install an application selling airport bus boarding passes.

[0084] In addition, the user's terminal device (100) can store acquired aircraft boarding pass data and airport bus boarding pass data.

[0085] The user's terminal device (100) can pre-register an automatic payment method on the system (1) for paying for airport bus tickets or aircraft tickets for each user's account. The automatic payment method may be a credit card issued in the user's name or a simple payment service subscribed to in the user's name.

[0086] The user's terminal device (100) can register the user's credit card as a payment method by inputting information necessary for card payment, such as card number, expiration date, and CVC, into a dedicated application of the system (1).

[0087] Alternatively, when the user's terminal device (100) inputs the user's simple payment service account (e.g., a user ID provided by the service) into the dedicated application of the system (1), the server (300) transmits a confirmation request to a third-party PG company system that provides the simple payment service, the third-party PG company system issues a payment linkage token for the user and transmits it to the server (300), and the server (300) can register the payment linkage token for the user as the user's payment method. The payment linkage token is a token having authentication information to which payment authority is granted. When the user attempts to pay for an airport bus ticket through the dedicated application, the server (300) notifies the third-party PG company system of the fact that "a specific user is making a payment through this token" and requests payment, and the third-party PG company system can approve the payment request based on the token.

[0088] The user's terminal device (100) can generate luggage bag image data by photographing the user's luggage bag to be loaded onto the aircraft. The user's terminal device (100) can generate bag interior image data by photographing the state in which the user's luggage bag is contained inside.

[0089] The user's luggage bag image and the bag interior image are described in more detail with reference to Fig. 7 below.

[0090]

[0091] In addition, the user's terminal device (100) can analyze the user's luggage bag image and the user's bag interior image to calculate luggage data that the user intends to entrust. The luggage data includes luggage item data obtained by analyzing the bag interior image and the user's luggage bag data obtained by analyzing the user's luggage bag image.

[0092] The user's luggage data may include luggage information indicating the result of recognizing which product or product group the user's luggage belongs to, and weight information of the recognized luggage. Also,

[0093] The above luggage bag data may include bag information of the luggage bag (i.e., recognition information) indicating the result of recognizing which bag product the user's luggage bag corresponds to, and weight information of the recognized luggage bag.

[0094]

[0095] The user's terminal device (100) and the airport bus's terminal device (200) may include one or more deep learning models for analyzing various images. The deep learning model has a neural network structure.

[0096] FIG. 2 is a schematic diagram of a neural network according to various embodiments of the present application.

[0097] A neural network (or referred to as an artificial neural network) can be composed of a set of interconnected computational units, which can generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node. The nodes (or neurons) constituting the neural network may be interconnected by one or more links. Within a neural network, one or more nodes connected via links may form a relative relationship between an input node and an output node. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the input node versus output node relationship can be generated around links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0098] In a relationship between an input node and an output node connected through a single link, the value of the output node's data can be determined based on the data input to the input node. Here, the links interconnecting the input and output nodes may have weights. These weights can be variable and may be adjusted by a user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node via respective links, the output node's value can be determined based on the values ​​input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.

[0099] As described above, a neural network consists of one or more nodes interconnected through one or more links, forming input-output node relationships within the neural network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​for the links, the two neural networks can be recognized as being different from each other.

[0100] A neural network can be composed of a set of one or more nodes. A subset of nodes constituting a neural network can form a layer. Some of the nodes constituting a neural network can form a layer based on their distances from an initial input node. For example, a set of nodes with a distance of n from an initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a way different from that described above. For example, a layer of nodes may be defined by its distance from a final output node.

[0101] Initial input nodes may refer to one or more nodes within a neural network to which data is directly input without passing through links in relation to other nodes. Alternatively, in terms of link-based relationships within the neural network, they may refer to nodes that do not have other input nodes connected by links. Similarly, final output nodes may refer to one or more nodes within a neural network that do not have output nodes in relation to other nodes. Furthermore, hidden nodes may refer to nodes constituting the neural network that are neither initial input nodes nor final output nodes.

[0102] A neural network according to one embodiment of the present disclosure may have the number of nodes in the input layer equal to the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. Additionally, a neural network according to another embodiment of the present disclosure may have the number of nodes in the input layer less than the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases as it progresses from the input layer to the hidden layer. Additionally, a neural network according to yet another embodiment of the present disclosure may have the number of nodes in the input layer greater than the number of nodes in the output layer, and may be a neural network in which the number of nodes increases as it progresses from the input layer to the hidden layer. A neural network according to yet another embodiment of the present disclosure may be a neural network in which the above-described neural networks are combined.

[0103] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to input and output layers. Using a deep neural network allows for the identification of the latent structures of data. That is, it is possible to identify the latent structures of photos, text, videos, voice, and music (e.g., what objects are present in a photo, what the content and emotions of a text are, what the content and emotions of a voice are, etc.). Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, Generative Adversarial Networks (GAN), restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, Generative Adversarial Networks (GAN), etc. The description of deep neural networks described above is merely illustrative and the present disclosure is not limited thereto.

[0104] Neural networks can be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The training of a neural network may be the process of applying knowledge to the neural network to perform a specific action.

[0105] Neural networks can be trained to minimize the error in their output. The training process involves repeatedly inputting training data into the network, calculating the error between the network's output and the target for the training data, and updating the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In supervised learning, training data is used where the correct answer is labeled for each individual training data point (i.e., labeled training data), whereas in unsupervised learning, the correct answer may not be labeled for each training data point. For instance, in the case of supervised learning for data classification, the training data may consist of data where each training data point is labeled with a category. The labeled training data is input into the neural network, and the error can be calculated by comparing the network's output (category) with the labels of the training data. As another example, in the case of unsupervised learning for data classification, the error can be calculated by comparing the input training data with the neural network's output. The calculated error is backpropagated in the neural network (i.e., from the output layer to the input layer), and through backpropagation, the connection weights of each node in each layer of the neural network can be updated. The amount of change in the connection weights of each node being updated can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's learning cycle. For example, a high learning rate can be used in the early stages of training to quickly achieve a certain level of performance and increase efficiency, while a low learning rate can be used in the later stages to improve accuracy.

[0106] In the training of neural networks, training data is generally a subset of real-world data (i.e., the data intended to be processed using the trained neural network); therefore, a training cycle may exist where errors decrease on the training data but increase on real-world data. Overfitting is a phenomenon in which errors on real-world data increase due to excessive training on the training data. For example, a neural network trained on cats by showing it yellow cats may fail to recognize cats other than yellow ones as cats, which can be a type of overfitting.

[0107] Overfitting can cause an increase in errors in machine learning algorithms. Various optimization methods can be used to prevent such overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (which disables some nodes in the network during training), and the use of batch normalization layers can be applied.

[0108]

[0109] To analyze an image, the user's terminal device (100) may include one or more deep learning models. The user's terminal device (100) may analyze the image using a specific deep learning model.

[0110] In various embodiments of the present application, the user's terminal device (100) may include a first deep learning-based recognition model that is machine-learned to recognize what kind of bag product the luggage bag shown in the luggage bag image is.

[0111] A deep learning-based recognition model is a machine learning model that has a deep learning neural network structure and parameters learned to recognize objects.

[0112] The above first deep learning-based recognition model is configured to receive an image of the user's luggage bag as input data and output bag information corresponding to the luggage bag appearing in the input luggage bag image as output data.

[0113] In some embodiments, the first deep learning-based recognition model may have a configuration similar to a deep learning-based identity verification model. The first deep learning-based recognition model may have various neural network structures capable of recognizing objects in an input image. For example, the first deep learning-based recognition model may have a Convolutional Neural Network (CNN) or other structures.

[0114] Additionally, in some embodiments, the first deep learning-based recognition model may be further configured to determine the bag type to which the luggage bag belongs as bag information and output the result of determining the bag type as bag information when it fails to determine bag information corresponding to the luggage bag, that is, when it fails to recognize what product the luggage bag is. The first deep learning-based recognition model may be configured to determine the bag type and product value for the luggage bag. If the product value is not determined, the first deep learning-based recognition model may output the recognition value of the bag type and the value of product recognition failure.

[0115]

[0116] In various embodiments of the present application, the user's terminal device (100) may include a machine-learned deep learning-based segmentation model to perform a segmentation operation that identifies and defines a sub-region (hereinafter, item area) in which luggage appears in an image inside a bag.

[0117] A deep learning-based segmentation model is a machine learning model that has a deep learning neural network structure and parameter values ​​learned to segment specific object regions in an input image.

[0118] The deep learning-based segmentation model described above can be machine-learned to perform semantic segmentation, instant segmentation, or panoptic segmentation operations.

[0119] Semantic segmentation is a task that separates objects by object type. In some embodiments, semantic segmentation is implemented as an operation that classifies a class for each pixel of the image to be segmented. In the semantic segmentation result, identical objects may be represented by the same color.

[0120] Instance segmentation is a task that divides objects by treating them individually. In some embodiments, instance segmentation may be implemented by labeling pixels in a region of interest (ROI) rather than labeling all pixels in the image to be segmented. Each object is masked in the instance segmentation result.

[0121] Panoptic segmentation is a type of segmentation that combines semantic segmentation and instance segmentation. Panoptic segmentation utilizes both instances (i.e., objects) and classes to segment an image and labels pixels based on instances of a class.

[0122] A deep learning-based segmentation model may have various neural network structures capable of segmenting object regions in an input image. For example, the deep learning-based segmentation model may be, for example, R-CNN (Regions with Convolutional Neuron Networks features), Faster R-CNN, YOLO (You only look once), FCN, SSD (Single shot detector), or other segmentation models.

[0123] The aforementioned R-CNN is a model that was the first to apply deep learning to the field of object detection, and it corresponds to a two-stage model that separates and executes the region proposal and object detection stages. YOLO is a deep learning model for detection that goes beyond modifying classification models and provides a new dimension of object detection methodology by defining object detection as a regression problem to calculate bounding box coordinates and the probability of each class. Unlike the R-CNN model, which separates and executes the region proposal and object detection stages, SSD is a deep learning model for detection that performs region proposal and object detection in a single step without separating them.

[0124] FCN has a structure that modifies CNN-based models. Unlike the basic VGG model, which attaches fully connected layers to the back of the network to extract image features, FCN is structurally characterized by placing CNNs instead of fully connected layers at the back of the network for segmentation. ParseNet is a deep learning model for segmentation that received high evaluations at the PASCAL-Context competition and is a modified version of the convolutional layers in FCN. DeconvNet (Convolutional and Deconvolutional Networks) is a deep learning model for segmentation that combines the convolutional network and inverse convolutional network of the VGG16 architecture. U-Net is a deep learning model for segmentation that demonstrates excellent performance even with a small amount of training data and features a U-shaped data processing path.

[0125]

[0126] In various embodiments of the present application, the user's terminal device (100) may include a second deep learning-based recognition model that is machine-learned to recognize what bag product a luggage item shown in an image inside a bag is.

[0127] A deep learning-based recognition model is a machine learning model that has a deep learning neural network structure and parameters learned to recognize objects.

[0128] In some embodiments, the second deep learning-based recognition model may have a configuration similar to the first deep learning-based recognition model.

[0129] The above second deep learning-based recognition model is configured to receive an image of the inside of the user's bag as input data and output luggage information corresponding to the luggage appearing in the input image of the inside of the bag as output data.

[0130] The above baggage information may be product information corresponding to the baggage. Product information is information that identifies baggage items for commercial purposes. In some embodiments, the above product information may include one or more of a product name, a product number, and a manufacturer.

[0131] The second deep learning-based recognition model may have various neural network structures capable of recognizing objects in an input image. For example, the second deep learning-based recognition model may have a Convolutional Neural Network (CNN) or other structures.

[0132] In various embodiments of the present application, the second deep learning-based recognition model may be a ViT model.

[0133] Additionally, in some embodiments, the second deep learning-based recognition model may be further configured to determine the item type to which the luggage belongs as product information and output the result of determining the item type as product information when it fails to determine product information corresponding to the luggage, that is, when it fails to recognize what product the luggage item is. The second deep learning-based recognition model may be configured to determine the item type and product value for the luggage. When the product value is not determined, the second deep learning-based recognition model may output the recognition value of the item type and the value of product recognition failure.

[0134] In various embodiments of the present application, the second deep learning-based recognition model may be further configured to recognize what product a luggage item with an incomplete shape appearing in an image inside the bag is and / or what item type (e.g., product group) it corresponds to. The second deep learning-based recognition model can recognize luggage regardless of whether the shape of the luggage appearing in the image inside the bag is complete or incomplete.

[0135] The above second deep learning-based recognition model can be machine-learned to recognize product information of luggage providing an incomplete shape by inferring a potential correlation between the features of luggage with an incomplete shape appearing in an image inside the bag and product information of luggage providing the incomplete shape.

[0136] The above-mentioned second deep learning-based recognition model will be described in more detail with reference to Figures 9 and 10 below.

[0137]

[0138] The user's terminal device (100) can calculate weight information of the recognized luggage bag based on the recognition result of the luggage bag. The user's terminal device (100) can calculate weight information of the recognized luggage items based on the recognition result of luggage items having a complete shape or an incomplete shape within the internal image of the bag.

[0139] The above user's terminal device (100) can generate user's luggage data including luggage bag information, luggage bag weight information, luggage item information for each luggage item in the luggage bag, and weight information for each luggage item.

[0140] The user's terminal device (100) can transmit to the server (300) an image of the user's luggage bag to be loaded onto the aircraft, an image of the inside of the user's bag, and data on the user's luggage.

[0141] The operation of such a user's terminal device (100) will be described in more detail with reference to Fig. 5 below.

[0142]

[0143] The terminal device (200) of the airport bus is a device including a processor, memory, communication unit, input device, and output device, and can be implemented in the form of a computer system such as, for example, a desktop computer, a laptop computer, a netbook, a tablet computer, an e-book reader, a GPS device, a camera, a personal information terminal (PDA), a POS terminal, a portable electronic device, a cellular phone, a smartphone, other computing device, other mobile device, other wearable device, other suitable electronic device, or any suitable combination thereof.

[0144] FIG. 3 is a configuration diagram of a terminal device of an airport bus according to various embodiments of the present application.

[0145] Referring to FIG. 3, the terminal device (200) of the airport bus may include a communication unit (201), a processor (202), a memory (203), a GPS sensor (204), and a camera (205). Additionally, in some embodiments, the terminal device (200) of the airport bus may further include a scanner (206). Additionally, in some embodiments, the terminal device (200) of the airport bus may further include an input device (207) and an output device (208).

[0146] The communication unit (201) is configured so that the terminal device (200) of the airport bus transmits and receives data via wired / wireless electrical communication with another external device (e.g., user's terminal device (100), or server (300)).

[0147] The communication unit (201) may include, for example, a cellular communication module, a short-range wireless communication module, or a GNSS (global navigation satellite system) communication module. The wireless communication unit (160) may establish a communication connection with a user's terminal device (100) or server (300) through a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct or IrDA (infrared data association), or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN).

[0148] The processor (202) may be configured to implement the procedures and / or methods proposed in the present invention.

[0149] The processor (202) may be implemented as at least one processor. The processor may include any combination of a CPU, a GPU (graphical processing unit), a single-core processor, a multi-core processor, an ASIC (application-specific integrated circuit), etc. In addition to the hardware implementation, the processor may be implemented in software and / or firmware. The software or firmware implementation of the processor may include computer- or machine-executable instructions that are described in any suitable programming language and perform the various functions described above. The software implementation of the processor may be stored in whole or in part in memory (203).

[0150] The processor (202) may be configured to control other components to implement some operations in the airport bus-linked checked baggage handling method, receive data from other components, perform calculation processing, and transmit the result to other components. For example, the processor (202) transmits or receives information, etc. through the communication unit (201). In addition, the processor (202) writes and reads data to and from memory (203).

[0151] The memory (203) can store programs of instructions that can be loaded onto the processor (202) and executed, and data generated during the execution of these programs. Examples of programs and data stored in the memory (203) may include an operating system that controls the operation of hardware and software resources available in the airport bus-linked baggage handling system (1), a driver for interacting with hardware devices such as a user's terminal device (100), an airport bus terminal device (200), and a server (300), a communication protocol for exchanging data with other hardware devices such as a user's terminal device (100) and a server (300), and additional software applications.

[0152] The memory (203) may be volatile (such as RAM) or non-volatile (such as ROM or flash memory). The memory (203) may provide storage of computer-readable instructions, data structures, program modules, and other data. The computer-readable medium may include at least two types of computer-readable media, namely, a computer storage medium and a communication medium.

[0153] The GPS sensor (204) is a component that detects the global positioning system (GPS) location of the airport bus. The GPS sensor (204) receives signals transmitted from three or more GPS satellites to determine the location of the satellites and the GPS sensor (204). By measuring the time difference between the signal transmitted from the GPS satellites and the signal received by the GPS sensor (204), the distance between the GPS satellites and the GPS sensor (204) can be calculated, and the signal transmitted from the GPS satellites contains information about the location of the GPS satellites. Once the distance to at least three GPS satellites and the location of each GPS satellite are known, the location of the GPS sensor can be calculated using a method such as trilateration. The terminal device (200) of the airport bus can determine the GPS location of the airport bus using the GPS sensor (204).

[0154] The GPS sensor (204) can be implemented in the form of a transmitter, receiver, or transceiver that communicates electrically using a protocol specialized for communication with GPS satellites.

[0155] The GPS sensor (204) can transmit GPS data having the GPS location information of the airport bus and the detection time to the processor (202). The processor (202) can perform an aircraft boarding intent authentication operation based on GPS coordinates based on the GPS data having the GPS location information of the airport bus.

[0156] The camera (205) is a component that photographs the face of a user boarding the airport bus. The camera (205) can be installed at a location inside the airport bus (e.g., the door) where it is easy to photograph the user's face.

[0157] The above camera (205) may be various imaging elements that recognize light and generate an image.

[0158] The camera (205) can capture the user's face and transmit the user's face image data generated to the processor (202). The processor (202) can perform an AI-based recognition operation based on the user's face image.

[0159] The scanner (206) is a component that scans a two-dimensional image corresponding to the mobile airport bus boarding pass displayed on the screen of the user's terminal device (100) in order to obtain information on the mobile airport bus boarding pass of the user boarding the airport bus.

[0160] The above two-dimensional image may be a QR code, barcode, or other readable image.

[0161] The terminal device (200) of the airport bus can determine whether the user has the right to board the bus based on the reading result of the scanner (206).

[0162] The input device (207) is a component configured to receive commands related to user input. The input device (207) may include a touch unit or other input units. A touch unit is a component in which a part of the user's body or another object is used as a pointing object to input user commands. The touch unit may include, but is not limited to, pressure-sensitive or electrostatic sensors. The other input units include, for example, buttons, keyboards, dials, switches, sticks, etc.

[0163] The output device (208) is a component that outputs information stored and / or processed in the terminal device (200) visually or audibly. The device that outputs visually (208) may include, for example, an LCD, an OLED, a flexible screen, etc., but is not limited thereto. The device that outputs audibly (208) may be, for example, a speaker, etc.

[0164] Although the input device (207) and the output device (208) are separated in FIG. 3, in many embodiments, the input device (207) and the output device (208) may be implemented as a single component to perform input reception and information output. For example, the input device (207) and the output device (208) may be a touch panel implemented as a touch screen forming a screen and a layer structure. Touch input is input by a pointing object (e.g., including the user's body or a tool).

[0165] Each component (201 to 208) of the terminal device (200) of the airport bus is connected to transmit / receive data to and from one another, but does not necessarily constitute a single physically integrated device. In various embodiments of the present application, the camera (205) is installed at a position having a shooting field of view suitable for photographing the face of a user boarding the airport bus, the scanner (206) is installed at a position suitable for placing a two-dimensional image displayed on the terminal device (100) of the user boarding the airport bus into the reading area of ​​the scanner (206), and the remaining components (201 to 204, etc.) may be implemented in a form included within a single device.

[0166] The terminal device (200) of the airport bus above can calculate the airport bus's GPS data through the GPS sensor (204).

[0167] The terminal device (200) of the airport bus can generate image data representing the face image of the user boarding the airport bus by photographing the user's face through a camera (205).

[0168] The terminal device (200) of the airport bus analyzes the user's face image through a processor (202) to calculate the user's identity data, and can authenticate the user's intention to board the aircraft based on the user's GPS data received from the terminal device of the verified user and the airport bus's GPS data.

[0169] The terminal device (200) of the airport bus can request and obtain the authenticated user's mobile aircraft boarding pass data from the user's terminal device through the processor (202) and the communication unit (201).

[0170] The terminal device (200) of the airport bus above can generate a baggage linkage code corresponding to the user's mobile aircraft boarding pass based on the user's baggage data of the mobile aircraft boarding pass data through a processor (202).

[0171] The terminal device (200) of the airport bus can transmit a linkage service reception message including the baggage linkage code to the user's terminal device via the server (300) by means of the communication unit (201).

[0172] In addition, to analyze images, the terminal device (200) of the airport bus may include one or more deep learning models.

[0173] In various embodiments of the present application, the terminal device (200) of the airport bus may include a deep learning-based identity verification model that extracts features from a face image of a user boarding the airport bus and verifies the identity of the user based on the extracted features.

[0174] The deep learning-based identity verification model described above is a machine learning model configured to recognize the identity information of a user having a face appearing in an input image by inferring a potential correlation between features extracted from a user's face image and the user's identity information.

[0175] In one example, a deep learning-based identity verification model may include an input layer, a feature extraction layer, a fully connected layer, and an output layer.

[0176] The input layer is a layer into which a face image is input, and it may be configured to input, for example, a 224x224 size image with RGB channels, but is not limited to this.

[0177] The feature extraction layer is a component configured to extract low-dimensional features from the input image. The feature extraction layer may include, for example, multiple convolutional layers and / or pooling layers. Convolutional layers consist of filters and activation functions to convert the input image into feature vectors. Generally, the ReLU activation function is used. Pooling layers are used to reduce the size of the feature maps extracted by the convolutional layers and to emphasize important features. Max pooling is primarily used.

[0178] The fully connected layer performs classification based on feature vectors from the pooling layer. Dropout technology is typically applied to the fully connected layer to prevent overfitting.

[0179] The output layer is a layer that calculates class probabilities that ultimately identify user identities from the computation results of the fully connected layer. Generally, the softmax function is applied to the output layer to calculate class probabilities.

[0180] The output data of the deep learning-based identity verification model output from the above output layer has a vector value containing a probability value for the user.

[0181] The above deep learning-based identity verification model can be implemented in the form of, for example, VGGNet, ResNet, InceptionNet, and other CNN models.

[0182] In one embodiment, the deep learning-based identity verification model may be implemented in the form of an on-device deep learning model that operates by a processor (202) and a memory (203) in a terminal device (200).

[0183] In another embodiment, the terminal device (200) of the airport bus may be configured to perform an identity verification operation through an API that can access a deep learning-based identity verification model. The deep learning-based identity verification model may operate on an external device (e.g., a third-party identity verification service provider).

[0184] The terminal device (200) of the airport bus above can verify the identity of a user using a deep learning-based identity verification model. The user identity verification information performed through the identity verification operation may include one or more of the user's name, gender, age, address, phone number, email, profile image, and other profile information.

[0185] When the airport bus arrives at the airport, the user's luggage bag, to which a luggage linkage code is assigned by the terminal device (200) of the airport bus, is transferred to a luggage processing space where a luggage inspection system (800) installed at the arrival airport is implemented.

[0186] The operation of the terminal device (200) of the airport bus is described in more detail with reference to Fig. 5 below.

[0187]

[0188] The employee terminal device (600) is a mobile terminal device that can communicate with the server (300) while moving with the employee in its possession at the boarding airport.

[0189] The employee providing user input from the employee terminal device (600) is the employee in charge of the checked baggage from the airport bus arriving at the boarding airport to the baggage inspection system inside the boarding airport during the airport bus connected checked baggage handling process.

[0190] The employee terminal device (600) may be configured to receive the user's luggage tag information from the server (300) and print a label sticker indicating the luggage tag. Additionally, the employee terminal device (600) may be configured to receive a luggage bag image indicating the luggage bag to which the label sticker is attached, along with the luggage tag information. The employee terminal device (600) may be configured to display the luggage tag information and the luggage bag image.

[0191] The above employee terminal device (600) is connected via wired or wireless connection to an external printer (600-1) as shown in FIG. 1, or is implemented in a form that includes a print unit inside the device.

[0192] Additionally, in various embodiments of the present application, the employee terminal device (600) may be further configured to function as an RFID reader for reading an RFID tag (700). To this end, the employee terminal (600) may be configured to be equipped with a reader module (600-2) for reading an RFID tag (700), or may be configured to read an RFID tag (700) by communicating with an external reader module.

[0193] To this end, the employee terminal device (600) may include some or all of a processor, memory, communication unit, printer, I / O interface, input device, and output device. Since these components are similar to the components of FIG. 3, a detailed description is omitted.

[0194] For clarity of explanation, embodiments of the present application are described using an employee terminal device (600) equipped with a reader module.

[0195]

[0196] The server (300) is configured to perform a linked checked baggage processing operation to transmit the user's baggage loaded onto the airport bus to the airport baggage inspection system (800) upon boarding, once the user's intention to board the aircraft is authenticated. To this end, the server (300) may be connected to communicate with the airport baggage inspection system (800) in addition to the user's terminal device (100), the airport bus terminal device (200), and the employee terminal device (600).

[0197] In certain embodiments, the server (300) may be a unitary server or a distributed server spanning multiple computers or multiple data centers. In some embodiments, the server (300) refers to a computer system and computer software (network server program) connected to a sub-device capable of communicating with other network servers via a computer network such as a private intranet or the Internet, receiving a request to perform a task, performing the task, and providing the result. However, the server (300) should be understood as a broad concept that includes, in addition to this network server program, a series of applications running on the server and, in some cases, various databases built internally. The server (300) may be of various types, such as, for example without limitation, a web server, a news server, a mail server, a message server, an advertising server, a file server, an application server, an exchange server, a database server, a proxy server, another server suitable for performing the functions or processes described herein, or any combination thereof. In certain embodiments, each server (300) may include hardware, software, or embedded logic elements or a combination of two or more such elements to perform appropriate functions implemented or supported by the server (300).

[0198] In various embodiments, the server may include one or more data stores (not shown) for storing data of the server. Data stores may be used to store various types of information. In some embodiments, information stored in data stores may be structured according to a specific data structure. Additionally, in some embodiments, each data store may be a relational, columnar, correlated, or other suitable database. Although this specification describes or illustrates specific types of databases, this specification considers any suitable type of database. In some embodiments, the server may provide an interface that enables managing, searching, modifying, adding, or deleting information stored in data stores.

[0199] The server (300) includes a user DB that manages user account information and usage history.

[0200] In various embodiments of the present application, the server (300) may include a bag DB that stores information about a plurality of luggage bags; and an item DB that stores information about items to be included in the luggage bags as luggage.

[0201] The bag database can store luggage bag type, product, manufacturer, weight, and other bag information for each bag.

[0202] The item database can store baggage type, product, manufacturer, weight, and other item information for each baggage item.

[0203] The server (300) can receive reading result data obtained from the luggage bag image of the user’s luggage bag taken from the luggage inspection system (800) and / or inspection source data for the luggage bag.

[0204] The reading result data may include an identification result for at least one luggage item contained in the user's luggage bag. The identification result is expressed as a product group or item type of luggage identified in the luggage bag through the security inspection device of the luggage inspection system (800). The product group or item type may correspond to a classification result of a class in the aforementioned second deep learning-based recognition model.

[0205] Additionally, the server (300) may receive the user's luggage data from the user's terminal device (100). Additionally, in some embodiments, the server (300) may further receive images of the user's luggage bag and images of the inside of the bag. The luggage data, etc., may be received in advance before receiving the luggage bag images taken of the user's luggage bag and / or the read result data obtained from the inspection source data for the luggage bag from the luggage inspection system (800).

[0206] Then, the server (300) may associate the reading result data for the luggage bag with the user's luggage linkage code based on the matching result of the luggage bag image of the luggage inspection system and the user's luggage bag image, the reading result data of the luggage inspection system and the matching result of the luggage item data in the user's luggage data, and transmit a notification message to the user's terminal device based on the message regarding the user's luggage inspection result received from the luggage inspection system.

[0207] The operation of such a server (300) will be described in more detail with reference to Figure 4 below.

[0208] Additionally, the server (300) includes a sales product / service DB, and the dedicated application of the system (1) can be further configured to sell various sales products / services registered in the DB.

[0209] The system (1) can provide affiliate services registered in the sales product / service DB at a discounted price to users who have paid for a bus ticket in real time through facial recognition. For example, when a foreigner arriving at Yangyang Airport uses the airport bus on the Yangyang Airport-Nami Island route and enters the Nami Island amusement park, the terminal device (200) of the amusement park recognizes the passenger's face through the camera 205 to verify the airport bus boarding history, and based on the verified history, can process the payment of a discounted admission fee using a pre-registered payment method.

[0210] For the above discount service, when real-time payment for a bus ticket is completed for a user account identified by facial recognition, the server (300) generates and stores bus boarding history information for the account and can determine whether to apply a discount for the affiliate service based on the stored bus boarding history information.

[0211]

[0212] In addition, the airport bus-linked checked baggage handling system (1) may further include an RFID tag (700) attached to the baggage bag.

[0213] FIG. 4 is a diagram of an RFID tag according to various embodiments of the present application.

[0214] Since the components (701, 702, 703) of FIG. 4 are similar to the components (201, 202, 203) of FIG. 3, the differences will be described in detail.

[0215] Referring to FIG. 4, the RFID tag (700) is configured to communicate with an RFID reader module. Through the RFID reader module, the employee terminal device (600) can communicate with the RFID tag (700) and provide tag information read from the RFID tag (700) to the employee.

[0216] The RFID tag (700) includes a communication unit (701), an RFID processor (702), and a memory (703).

[0217] The communication unit (701) includes an RFID reader module and an antenna that transmits and receives signals wirelessly. For example, the communication unit (701) may be an antenna that communicates with LF band signals and converts the received RF signal into an electrical signal or converts the electrical signal into an RF signal and transmits it.

[0218] The RFID processor (702) includes an analog part that processes an analog signal received from an antenna (701) or converts digital information received from a digital part into an analog signal, and a digital part that converts the signal received from the analog part into digital information or retrieves digital information stored in a memory (703) and transmits it to the analog part. To this end, the RFID processor (702) may be composed of a combination of various types of analog processors and digital processors.

[0219] The memory (703) is a space for storing various information related to the RFID tag (700). The memory (703) stores a UID for identifying the RFID tag (700) and various tag information read by an RFID reader module. The memory (703) includes a space for the UID and a space for tag information.

[0220] The above tag information may be the user's baggage tag information or UID and other information.

[0221] The initialized RFID tag (700) has no tag information stored yet and only stores the corresponding UID. The user's luggage tag information can be stored in the initialized RFID tag (700) as the tag information of the RFID tag (700).

[0222] The RFID tag (700) may be in the form of a card or in various forms made of plastic or other flexible materials, but is not limited thereto.

[0223] The RFID tag (700) can be placed on a luggage bag. In some embodiments, the RFID tag (700) can be attached to a label sticker. A sticker can be attached to the back of the RFID tag (700). Thus, the RFID tag (700) can be easily attached to a luggage bag that requires tracking.

[0224] When the employee terminal device (600) transmits the user's luggage tag information to the RFID tag (700) as new tag information through the reader module, the new tag information can be stored in the memory (703).

[0225] The above RFID reader module can read tag information stored in memory (703). If the user's luggage tag information is already stored in the RFID tag (700), the user's luggage tag information is read and obtained.

[0226] In various embodiments of the present application, the RFID tag (700) may be a passive type that does not require its own power supply. The passive RFID tag (700) may transmit the user's luggage tag information stored in advance to an employee terminal device (600) or another external terminal device through an RFID reader module.

[0227] Additionally, the RFID tag (700) may be configured to perform various encryption and / or verification methods to ensure the reliability of the stored tag information.

[0228] The operation of such RFID tags (700) will be described in more detail with reference to Fig. 5 below.

[0229]

[0230] Specifically, the RFID reader module may perform the following steps: transmitting a wireless signal to a passive RFID tag (700) to activate the RFID tag (700); transmitting tag information stored in the activated RFID tag (700) as a response signal; transmitting the tag information of the received RFID tag (700) to an employee terminal device (600); comparing or processing the transmitted tag information with the user's luggage tag information stored in the server (300) at the employee terminal device (600); transmitting the comparison processing result of the RFID tag (700) to the server (300) to update the user's usage history - updating the location and status of the luggage in real time; and if the comparison processing result of the RFID tag (700) is abnormal, searching for the user associated with the RFID tag (700) and transmitting a luggage status guidance message describing the abnormal state of the RFID tag (700) to the terminal device (100) of the searched user.

[0231] Through this series of processes, the user's luggage tag information stored in the passive RFID tag (700) is accurately read and managed, and if an abnormal condition is detected, a rapid response is possible through immediate notification.

[0232] These RFID tags (700) are used to improve the reliability of the baggage handling service provided by the system (1). The operation of these RFID tags (700) is described in more detail with reference to Fig. 5 below.

[0233]

[0234] Referring again to FIG. 1, the baggage inspection system (800) is an inspection system installed at the airport where the user boards (i.e., the boarding airport). The baggage inspection system (800) is configured to handle a series of baggage processes for loading baggage onto an aircraft at the boarding airport.

[0235] Specifically, the baggage inspection system (800) can detect checked baggage corresponding to restricted baggage in the user's baggage bag read based on a preset restricted baggage guideline.

[0236] The above checked baggage restriction guidelines may consist of information on items with capacity restrictions and information on checked baggage restrictions. These checked baggage restriction guidelines may be identical to the checked baggage restriction guidelines of Fig. 11 below.

[0237] The above baggage inspection system (800) can be implemented by at least one component among a plurality of computing devices, a conveyor belt system, and a means of transportation.

[0238] Specifically, the baggage inspection system (800) may include a baggage scanner, a security inspection device that controls the baggage scanner and sees through the inside of the baggage bag at the baggage scanner, an output device that outputs the inspection results of the security inspection device, and / or a reading device that reads the inspection results of the security inspection device.

[0239] A security inspection device may be a variety of devices for discovering or detecting restricted items in checked baggage inside a baggage bag by inspection personnel or computer algorithms.

[0240] The security inspection device provides inspection source data used to generate reading results for inspection staff or readers to find or detect restricted items in checked baggage.

[0241] For example, the security inspection device may be any one of an X-RAY scanner, a CT scanner, a 3D CT scanner, an EDS (Explosive Detection Systems) device, an ETD (Explosive Trace Detection) device, a neutron scanner, a gamma ray scanner, a large cargo scanner, and a combination of two or more of these, but is not limited thereto.

[0242] X-ray scanners can generate X-ray images by penetrating the interior of luggage with X-rays. X-ray images are used to visually identify foreign or prohibited items (explosives, weapons, etc.).

[0243] CT scanners are an advanced version of X-ray technology that can be used to analyze luggage into 3D images.

[0244] EDS and EDT devices are instruments that detect small particles and chemical components of explosives contained in baggage. EDS and EDT devices can be used to assist other security screening equipment.

[0245] Large cargo scanners can be used to inspect not only checked baggage but also cargo on cargo planes.

[0246] The reading device may be configured to identify the item type (or product group) of the user's luggage contained within the luggage bag through various reading algorithms that analyze inspection results. To this end, the reading device may be configured to detect the user's checked luggage corresponding to restricted checked luggage in the user's luggage bag through various image analysis software or deep learning-based restricted luggage detection programs known at the time of filing of this patent.

[0247] For example, the baggage inspection system (800) may detect specific baggage items belonging to a user that correspond to restricted checked baggage in a user's baggage bag by utilizing the technology described in Prior Art 1 (Patent Registration Publication No. 10-2063859 (published on February 11, 2020, AI and deep learning-based airport security screening system and method), but this is merely illustrative. The reading technology of the reading device is not limited thereto and may provide reading result data by utilizing other automatic reading technologies.

[0248] Alternatively, the baggage inspection system (800) may generate reading result data based on user input from an inspection officer. User input from an inspection officer may indicate the location of a user's baggage item suspected of being restricted baggage on a security inspection image.

[0249] The baggage inspection system (800) can transmit the baggage inspection results to the server (300) based on whether the user's checked baggage corresponding to restricted checked baggage is detected in the user's baggage bag.

[0250] In various embodiments of the present application, the baggage inspection system (800) can generate a rejection message for baggage inspection and transmit it to the server (300) when a user's checked baggage corresponding to restricted checked baggage is detected in the user's baggage bag.

[0251] The above rejection message includes the baggage linkage code of the baggage bag in which the restricted checked baggage was detected, location information of a place where the baggage bag that failed baggage inspection is temporarily stored or where the user can visit to check the baggage, and message content indicating failure to pass baggage inspection. In some embodiments, the rejection message may further include information on the item detected as restricted checked baggage.

[0252] Additionally, if the baggage inspection system (800) does not detect any checked baggage belonging to the user that is restricted from checked baggage in the user's baggage bag, it may issue a baggage management number for the user's baggage bag, generate a pass message for the baggage inspection including the baggage management number for the baggage baggage, and transmit the pass message to the server (300).

[0253] It includes a baggage management number for the user's baggage bag and a message indicating successful passage of the baggage inspection.

[0254] The operation of this baggage inspection system (800) will be described in more detail with reference to Fig. 4 below.

[0255] It will be apparent to a person skilled in the art that the above airport bus-linked checked baggage handling system (1) may include other components not described herein to implement the embodiments. For example, the airport bus-linked checked baggage handling system (1) may include other hardware elements necessary for the operation described herein, such as an input device for data entry and an output device for printing or other data display, and driving-related components such as wheels and links.

[0256]

[0257] Checked baggage handling services provided at the boarding stage

[0258] According to another aspect of the present application, an airport bus-linked checked baggage handling method that utilizes AI-based recognition and GPS coordinates and enhances reliability through RFID (hereinafter, airport bus-linked checked baggage handling method) can be performed by an airport bus-linked checked baggage handling system (1) that implements baggage linkage proceeding along the checked baggage route from the airport bus to the aircraft through the airport baggage inspection system as described above.

[0259] The airport bus-linked checked baggage handling method is for users at the airport

[0260] FIG. 5 is a flowchart of an airport bus-linked checked baggage handling method according to various embodiments of the present application.

[0261] Referring to FIG. 5, the airport bus-linked checked baggage handling method includes the step (S10) of preparing an RFID tag (700) initialized in the system (1).

[0262] The system (1) may be equipped with a number of RFID tags (700) that can be attached to each of the luggage bags of users boarding the airport bus. All of the RFID tags (700) store a UID, but are in an initial state in which no tag information is stored yet.

[0263] Additionally, the above airport bus-linked checked baggage processing method includes a step (S100) of calculating the user's baggage data by analyzing the user's luggage bag image and the bag interior image captured by the user's terminal device (100).

[0264] The luggage bag image is a photograph of a user's luggage bag to be loaded onto the aircraft. The luggage bag image displays various appearance information of the luggage bag. For example, the luggage bag image displays the shape, color, texture, and other appearance information of the luggage bag. Additionally, the luggage bag image may display trademarks and product-related information.

[0265] The image of the bag interior shows the user's baggage items placed inside the interior space of the bag.

[0266] The luggage data includes luggage bag data (or information) describing the luggage bag, and luggage item data describing the luggage items contained in the luggage bag. The interior images of the bag display various external information of the luggage items. The interior images of the bag are described in more detail with reference to Figure 7 below.

[0267] FIG. 6 is a detailed flowchart of a process for calculating user luggage data by analyzing the user’s bag exterior image and bag interior image according to various embodiments of the present application.

[0268] Referring to FIG. 6, the step of calculating the user's luggage data (S100) includes: a step of generating an image of the user's bag exterior by photographing the user's bag exterior and generating the user's luggage bag data by analyzing the photographed image of the user's bag exterior (S110); and a step of generating an image of the user's bag interior by photographing the state in which the user's luggage items are contained in the interior space of the bag and generating the user's luggage item data by analyzing the photographed image of the user's bag interior (S130).

[0269] The user's terminal device (100) can generate the user's luggage data by analyzing the bag interior image and luggage bag image using one or more deep learning models.

[0270] The step of calculating the luggage bag data (S110) includes: a step of recognizing the user's luggage bag by applying the user's luggage bag image to a first deep learning-based recognition model (S111); a step of obtaining weight data of the user's luggage bag through a search engine or server (300) based on the recognition result of the luggage bag (S115); and a step of generating luggage bag data including recognition information and weight data of the luggage bag (S117).

[0271] The user's terminal device (100) can produce a recognition result of the luggage bag, i.e., recognition information, indicating what product or bag type the luggage bag shown in the user's luggage bag image is (S111). In some embodiments, the recognition information of the luggage bag may include one or more of the bag type, bag manufacturer, and bag product number of the bag product.

[0272] The first deep learning-based recognition model is a deep learning-based recognition model that has been pre-trained to output recognition information of a luggage bag appearing in a luggage bag image. As the first deep learning-based recognition model has been described above, a detailed explanation is omitted.

[0273] A search engine is a software system that quickly finds the information users want from large amounts of data. It performs key functions such as collecting information through crawling, classifying and storing information through indexing, and extracting relevant results through searching. For example, web search engines like Google or Naver crawl web pages to collect information, index it to enable efficient searching, and then provide highly relevant web pages based on the user's search terms. Terminal devices utilize search engines to perform processes such as generating search queries, transmitting search requests, and processing search results, thereby enabling the automatic acquisition of accurate weight information for luggage bags.

[0274] The user's terminal device (100) generates a search query to search for the weight of the luggage bag based on the recognition information of the luggage bag, and applies the search query to the bag DB of the search engine or server (300) to obtain the weight information of the luggage bag of the search query (S115). The search query contains recognition information of the luggage bag. For example, if the luggage bag is recognized as a specific model of a specific manufacturer, a search query including the manufacturer's name and model name may be generated. The search engine may use this search query to search for weight information from the manufacturer's website, online shopping mall, or the bag DB of the server (300) that contains detailed information about the bag. Alternatively, the average weight information of bags with similar specifications may be obtained through a search query that includes the type of bag (e.g., hard case, soft case) and size information. To increase the accuracy of the search results, the search engine can select the most appropriate weight information by utilizing product information in combination with the visual features of the recognized bag. The user's terminal device (100) generates a search query to search for the weight of the luggage bag based on the recognition information of the luggage bag, and applies the search query to the bag DB of the search engine or server (300) to obtain the weight information of the luggage bag of the search query (S115). The search query has recognition information of the luggage bag.

[0275] For example, if the result of recognizing the luggage bag from the luggage bag image is "American Tourister ROCKFORD69 Carrier BG941002", the user's terminal device (100) can search for information about the luggage bag through the bag DB of the search engine or server (300) and obtain the weight of the bag "3.96kg" from the information about the luggage bag.

[0276] The user's terminal device (100) generates luggage bag data including bag recognition information "American Tourister ROCKFORD69 Carrier BG941002" and bag weight information "3.96kg" (S117).

[0277] The step of generating the luggage item data (S130) comprises: applying the user’s bag interior image to a deep learning-based segmentation model to segment the item area where the luggage item appears in the user’s bag interior image, thereby extracting a sub-image representing the item area (S131); calculating size information for at least one luggage item among the luggage items appearing in the bag image based on the recognition result of the user’s luggage bag and the segmentation result of the item area (S132); applying the extracted sub-image of the item area and the size information for at least one luggage item among the luggage items appearing in the bag interior image to a second deep learning-based recognition model to recognize at least one luggage item appearing in the user’s bag interior image (S133); calculating the user’s luggage item data by obtaining weight data of the luggage item contained in the user’s luggage bag through a search engine or server (300) based on the luggage item recognition result (S135); and generating luggage item data including the luggage item recognition result and weight information for each luggage item (S137).

[0278] FIG. 7 illustrates an image of the inside of a luggage bag taken according to various embodiments of the present application. FIG. 8 illustrates the result of dividing clothing items included in a luggage bag according to various embodiments of the present application.

[0279] Referring to FIG. 7, an image of the inside of a user's bag captured by the user's terminal device (100) shows a state in which one or more luggage items are contained in the luggage bag. The luggage items shown in the image of the inside of the bag may consist of luggage items with a complete shape without partial obscuration and / or luggage items with an incomplete shape.

[0280] Luggage with an incomplete shape may be partially obscured by other luggage, or may be a modified version of luggage with a complete shape. The interior image of the bag may include luggage having an incomplete shape due to deformation of the shape, or luggage partially obscured by other luggage located above it.

[0281] The luggage items shown in the above-mentioned image of the bag interior may be various pouches, shoes, and other travel items, including folded clothing, hats, underwear, accessories, socks, medicines, daily necessities, and / or small travel items. Folded luggage items are treated as luggage items with an incomplete shape.

[0282] As shown in FIG. 7, the interior image of the bag may show incomplete luggage items, such as clothing (I1, I2, I4, I5) and a hat (I3), which have a shape deformed to fit the interior space of the luggage bag, and / or luggage items having a complete shape, such as sandals. Additionally, in FIG. 7, the pouch may have an incomplete shape because it is partially obscured by the sandals, and clothing such as a skirt or hat may have an incomplete shape because it is folded or deformed.

[0283] In the above step (S131), the user's terminal device (100) can input an image of the inside of a bag as shown in FIG. 7 into a deep learning-based segmentation model. When the user's image of the inside of the bag is input, the deep learning-based segmentation model can be machine-learned to infer the correlation between the characteristics of the luggage and the item type of the luggage in the image of the inside of the bag, identify the item type of the luggage appearing in the image of the inside of the bag, and identify the item area, which is a sub-area where the luggage appears in the image of the inside of the bag.

[0284] The deep learning-based partitioning model described above may have various neural network structures capable of performing partitioning operations as described above.

[0285] The deep learning-based segmentation model above has an image of the inside of a bag as an input image.

[0286] The deep learning-based segmentation model above has item regions in the bag interior image as output data. The item regions are represented as the result of labeling the pixels in the bag interior image where the corresponding item appears as the class of the corresponding item.

[0287] In the above deep learning-based partitioning model, the class is determined by the partitioning method. For example, in a deep learning-based partitioning model where semantic partitioning is applied as the partitioning method, the class can be an item type.

[0288] In various embodiments of the present application, the deep learning-based segmentation model may be trained using a first training data set. The first training data set consists of a plurality of training samples, each training sample consisting of training data and label data.

[0289] The above training data may be at least one training image.

[0290] At least one training image may be a sample image of an incomplete shape representing a training sample item (e.g., clothing) having an incomplete shape. In some embodiments, the incomplete shape shown in the at least one training image may be a shape according to the state in which it is received in a luggage bag.

[0291] In some embodiments, the training data may further include a sample image of a complete shape, which represents the luggage item having a complete shape as the luggage item shown in the sample image of the incomplete shape.

[0292] The label data represents the actual values ​​of the item types of the luggage shown in the corresponding training images.

[0293] When training a segmentation model using the first training data set, the segmentation model can learn to recognize luggage items with incomplete shapes based on features of the boundary portions of complete luggage items appearing in sample images of complete shapes, features of the boundary portions of incomplete luggage items appearing in sample images of incomplete shapes, and features of the appearance difference between the incomplete and complete shapes of the same luggage items, and to segment the sub-region in the input image where luggage items with incomplete shapes appear.

[0294] In some embodiments, the deep learning-based segmentation model may be trained to recognize luggage items with incomplete shapes by giving more weight to features of the boundary portions of luggage items with incomplete shapes. As a result, the deep learning-based segmentation model can accurately detect item regions where luggage items with incomplete or complete shapes appear.

[0295] When the bag interior image of FIG. 7 is input to the deep learning-based segmentation model learned in the above step (S131), the user's terminal device (100) can segment a plurality of item regions representing clothing luggage in the bag interior image through the deep learning-based segmentation model as shown in FIG. 8 (S131).

[0296]

[0297] In some embodiments, the first training data set may be generated by a server (300). The server (300) may generate the first training data set to train a deep learning-based segmentation model and provide the trained deep learning-based segmentation model to a user's terminal device (100).

[0298] Specifically, the server (300) may extract one or more sub-regions from an item region in which an item with a complete shape appears in a sample original image in which an item with a complete shape appears, and generate a first training data set having images of the extracted sub-regions as training images. A sub-region represents a part of the shape of the complete shape. Among the one or more sub-regions extracted from the original image of the same item, some sub-regions may represent different parts of the item from other sub-regions.

[0299] Through a deep learning-based segmentation model trained using this first training data set, the user's terminal device (100) can obtain an item area as a segmentation result and extract the item area from the bag interior image.

[0300]

[0301] Meanwhile, the user's terminal device (100) can extract a sub-image including one or more item regions from a single bag interior image based on the division result (S131). If multiple luggage items are included in the bag interior image, a set of sub-images for each of the multiple luggage items is extracted. The sub-image of the item region is a sub-image including the divided item region among the bag interior images, and is used as an input image to be input to a second deep learning-based recognition model.

[0302] In various embodiments of the present application, the step of extracting the sub-image (S131) ​​may include, prior to applying to a second deep learning-based recognition model, a step of calculating the vertical relationship between items appearing in the divided item area; a step of selecting item areas separated from each other in which items having a vertical relationship with respect to a specific item appear separately; a step of detecting attribute information of the selected item areas; a step of determining item areas representing the same item among the separated item areas based on the attribute information of the selected item areas as item areas representing the same item; and a step of extracting a single sub-image including the separated item areas.

[0303] The user's terminal device (100) can select adjacent item regions among the divided item regions and determine the vertical relationship between adjacent items appearing in each adjacent item region based on one or more of the brightness distribution, the area of ​​the shadow portion, the shape of the shadow portion, and the shape of the item in the adjacent item regions.

[0304] In the above step (S131), the user's terminal device (100) can analyze the vertical relationship between the floral skirt (I1) and the white top (I4) and determine that the floral skirt (I1) is positioned above the white top (I4).

[0305] In the above step (S131), the user's terminal device (100) selects different item areas (A4) representing a white top (I4) and can calculate attribute information of the corresponding area by analyzing the two selected item areas (A4). The attribute information of the area may include one or more of brightness, texture, color, and pattern.

[0306] The user's terminal device (100) can determine the item areas (A4) representing the same item, a white top (I4), among the separated item areas based on the attribute information of the luggage items as the same item area.

[0307] Then, the user's terminal device (100) can extract a single sub-image including separated item areas (A4) as a sub-image for the white top (I4) (S131).

[0308] Referring again to FIG. 6, the step (S132) of calculating size information for at least one luggage item; can calculate size information for a user’s luggage bag recognized in step (S110) in a search engine or server (300); and size information for luggage items contained in the luggage bag based on size information for the user’s luggage bag, size information for a luggage bag area in an internal bag image, and size information for a divided item area. The size of the luggage bag area is the size of a sub-area divided into luggage bag areas in an internal bag image.

[0309] If the result of recognizing the luggage bag from the luggage bag image in step (S110) is "American Tourister ROCKFORD69 Carrier BG941002", the user's terminal device (100) can search for information about the luggage bag through the bag DB of the search engine or server (300) and obtain information about the width, height, and depth of the luggage bag from the information about the luggage bag.

[0310] The size information of the baggage items includes the width and height obtained through the baggage's bounding box.

[0311] The user's terminal device (100) calculates size information of the luggage item when it has an incomplete shape (S132). In FIG. 7, size information of the folded state of the clothes can be calculated (S132). In addition, the size information of the luggage item that is partially covered by being stacked underneath is the size information of the exposed part.

[0312] The user's terminal device (100) can input the sub-image extracted in step (S131) ​​as an input image to the second deep learning-based recognition model (S133). Additionally, the user's terminal device (100) can input size information of the luggage item shown in the sub-image calculated in step (S132) into the second deep learning-based recognition model.

[0313] The second deep learning-based recognition model is configured to recognize what kind of item the specific luggage item is based on a sub-image of the input specific luggage item and size information of the specific luggage item.

[0314] Sub-images and size information for the same item are simultaneously input to the second deep learning-based recognition model. If size information for an item is not calculated, a sub-image representing the baggage item and size information having a value representing the uncalculated item may be input. In this case, the second deep learning-based recognition model recognizes the baggage item by analyzing only the sub-image of the specific baggage item.

[0315] FIG. 9 is a model network structure diagram of a second deep learning-based recognition model according to some embodiments.

[0316] Referring to FIG. 9, the second deep learning-based recognition model may include an input layer (810), a preprocessing layer (820), a feature extraction layer (830), a fully connected layer (850), and an output layer (870). As the input layer (810), the feature extraction layer (830), the fully connected layer (850), and the output layer (870) have been described above, they will be described mainly in terms of their differences.

[0317] The input layer (810) is a first input layer, which is a layer into which a sub-image of the item area extracted in step (S131) ​​is input. The first input layer (810) may be configured to input, for example, a 224x224 size image with RGB channels, but is not limited thereto.

[0318] Sub-image data input through the first input layer (810) is input to the feature extraction layer (830).

[0319] The preprocessing layer (820) is a second input layer configured to receive size information of the luggage item calculated in step (S135) and normalize it based on the size of the sub-image.

[0320] In some embodiments, the second deep learning-based recognition model can calculate the normalized size information (w', h') of the baggage appearing in the item area through the following mathematical formula.

[0321] [Mathematical Formula 1]

[0322] w' = w / W

[0323] h' = h / H

[0324] Here, w and h represent the width and height of the luggage calculated in step (S132), respectively, and W and H represent the height and size of the image.

[0325] The normalized size information of the luggage processed in the second input layer (820) is combined with the feature extraction result of the feature extraction layer (830).

[0326] The feature extraction layer (830) is configured to extract features from a sub-image of an item area. The extracted features can be implemented in the form of a feature map. The information contained in the feature map represents features useful for recognizing luggage items having a state (e.g., incomplete shape) contained in a luggage bag.

[0327] In various embodiments of the present application, the feature map may include features extracted from the boundary portion of the corresponding baggage in the sub-image.

[0328] The second deep learning-based recognition model combines the value of the size information of the baggage item normalized in the preprocessing layer (820) with the feature (i.e., feature map) extracted from the feature extraction unit 830, and inputs the combined result to the fully connected layer (850).

[0329] In some embodiments, the second deep learning-based recognition model may be configured to convert normalized baggage item size information into a size information tensor, generate baggage tensor data based on a feature map and a size information tensor extracted from a sub-image, and input the generated baggage tensor data to a fully connected layer (850) as an intermediate operation result.

[0330] If the size of the feature map is (C, Hf, Wf), the size information tensor can be expressed in the form (2, w', h').

[0331] The above second deep learning-based recognition model combines the size information tensor with the feature map of the sub-image at the channel level. Then, the combined baggage tensor data represents a final tensor in which two channels (size information) are added to the existing number of channels in the feature map of the existing sub-image.

[0332] The above-mentioned fully connected layer (850) is trained to recognize what product or product group the luggage item shown in the sub-image is by receiving and processing the tensor data of the luggage item, inferring potential correlations between the luggage item's size information, feature information, and the state of the luggage item contained in the luggage bag (e.g., incomplete shape). The feature information may represent features of the luggage item with an incomplete shape, such as partial occlusion or deformation.

[0333] Through the above-mentioned fully connected layer (850), the second deep learning-based recognition model can recognize what kind of product or product group the luggage item shown in the sub-image having the incomplete shape is by considering the characteristics of the luggage with the incomplete shape.

[0334] The above-mentioned fully connected layer (850) can calculate the probability that the luggage item in the sub-image corresponds to a certain product.

[0335] The output layer (870) can output a recognition result of a baggage item based on the operation result, i.e., the inference result, of the fully connected layer (850). The recognition result is output as product information or product group information. The product group may be a product type.

[0336] The recognition result may be product information of the actual baggage or product information of similar baggage recognized as being most similar to the actual baggage.

[0337] In some other embodiments, the second deep learning-based recognition model may be a ViT model. The ViT (Vision Transformer) model is an application of the Transformer model originally used for natural language processing to image recognition, which converts image patches into a series of tokens and inputs them into a Transformer encoder to learn the relationships between the patches.

[0338] The second deep learning-based recognition model comprises an input layer that inputs the sub-image as an input image, an encoder including one or more encoding blocks, a decoder including one or more decoding blocks, and a classification layer that recognizes and outputs luggage items appearing in the sub-image based on an encoding vector output from the decoder.

[0339] The above second deep learning-based recognition model may be configured to divide an input sub-image into a plurality of patches, convert each patch into a patch embedding vector, combine a position embedding value representing the position of each patch with the patch embedding vector of the corresponding patch, input the combined vector into the encoder to process and produce an encoding vector, recognize luggage items appearing in the sub-image based on the encoding vector, and output a recognition result.

[0340] The input sub-images in the ViT model can be multiple smaller patch images (e.g., 16x16 pixels). Each patch is converted into a fixed-length vector, which has a form similar to tokenized text.

[0341] Through patch embedding, each patch vector is mapped to a high-dimensional space, and positional encoding representing the patch's location within the sub-image is added to reflect the patch's positional information in the patch vector.

[0342] In the ViT model, the Transformer encoder is input with a sequence combining patch vectors and position encoding. The encoder learns the relationships between patches through a Multi-head Self-Attention mechanism. In this process, it assigns more weight to patches that are relatively important for recognizing luggage items appearing in sub-images, thereby overcoming incomplete shapes such as partial occlusion and learning the ability to recognize objects.

[0343] Specifically, the Transformer encoder can be implemented by stacking multiple encoder layers. These encoder layers may consist of self-attention and feed-forward layers. Self-attention is used to determine the relationships between words, and it automatically calculates weights indicating how each word influences the others. The embedding vector input to the prompt transformation model is input into the encoder's first encoding layer.

[0344] In some embodiments, the encoder layer may be composed of a multi-head attention layer and a feed forward layer.

[0345] Multi-head attention is divided into multiple attention heads, each capable of extracting specific information from a given sequence. If there are h attention heads, each token in the token embedding is analyzed into h different pieces of information. This analysis operation plays a role similar to filters in convolutional neural networks in image processing. The analysis into h different pieces of information is analogous to a case in a convolutional neural network where one filter processes a circle and another processes a square. A single attention head calculates a weight matrix and then calculates a weighted average for each token in the token embedding; here, the weight matrix can be referred to as the attention score. Through this process, a similarity score is calculated between each token in the token embedding. Tokens with high association receive a large score, while those with low association receive a small score. By increasing the score of the parts to be emphasized (attention) in this way, the attention mechanism is applied, and a self-attention operation is performed where scores are calculated for identical token embeddings.

[0346] An attention head can be implemented by using token embeddings to generate and compute query vectors (Q), key vectors (K), and value vectors (V) through a fully connected layer. A weight matrix is ​​created using the dot product (or MatMul) of the query vector and key vector; this matrix is ​​then normalized and a softmax function is applied to ensure the column sum is 1; finally, the weight matrix is ​​multiplied by the value vector to produce the final result. Meanwhile, when training to predict recommendation prompts, an additional step is performed before the softmax function of the weight matrix to set the positions of the next tokens to negative infinity values, preventing the current token from referencing the next tokens (i.e., tokens representing recommendation prompts). This additional step can be referred to as masking. Stacking multiple such attention heads creates multi-head attention.

[0347] A feed-forward layer consists of multiple fully connected layers. For example, a feed-forward layer can consist of two fully connected layers.

[0348] In the above ViT, the encoder transformer outputs a class token. The class token represents summary information of the image. The class token of the above ViT is input into the classification layer.

[0349] The above classification layer recognizes what kind of product or product group the baggage item is based on class tokens.

[0350]

[0351] In various embodiments of the present application, the second deep learning-based recognition model may be trained using a second training data set. The second training data set consists of a plurality of training samples, each training sample consisting of training data and label data.

[0352] The above training data may be at least one training image.

[0353] At least one training image may be a sample image of an incomplete shape representing a training sample item (e.g., clothing) having an incomplete shape. In some embodiments, the incomplete shape shown in the at least one training image may be a shape according to the state in which it is received in a luggage bag.

[0354] In some embodiments, the training data may further include a sample image of a complete shape, which represents the luggage item having a complete shape as the luggage item shown in the sample image of the incomplete shape.

[0355] The label data represents the actual values ​​of the item types of the luggage shown in the corresponding training images.

[0356] When training a recognition model using the second training data set mentioned above, the recognition model can learn to recognize luggage items with incomplete shapes by using features of the boundary portions of complete luggage items appearing in sample images of complete shapes, and features of the boundary portions of incomplete luggage items appearing in sample images of incomplete shapes.

[0357] In some embodiments, the second deep learning-based recognition model may be trained to recognize luggage items with incomplete shapes by giving more weight to features of the boundary portions of luggage items with incomplete shapes.

[0358] Additionally, in the second deep learning-based recognition model of FIG. 9, the feature extraction layer (830) may include a first encoder (831) and a decoder (835). The first encoder (831) and the decoder (835) may be learned by interacting with the second encoder (832). The first encoder (831) corresponds to the encoder in ViT, and the decoder (835) corresponds to the decoder in ViT.

[0359] FIG. 10 is a flowchart of the learning process of a second deep learning-based recognition model including a first encoder and a decoder, according to specific embodiments of the present application.

[0360] Referring to FIG. 10, the second training data set consists of a plurality of training samples. Each training sample includes a pair of sample images.

[0361] The above pair of sample images is an incomplete shape sample image representing a learning sample item (e.g., clothing) having an incomplete shape, and a complete shape sample image representing the complete shape of the same luggage item.

[0362] The above-mentioned complete shape sample image may be an image provided by a seller for sale on an online shopping mall or smart store (e.g., a front image or a side image of a product). For example, it may be a product image of clothing with a non-folding shape (I1, I2, etc.). Or, it may be a product image of clothing accessory (I3) with a wrinkle-free shape.

[0363] The above sample image of the incomplete shape represents a shape identical or similar to the shape according to the state in which it is received in a luggage bag.

[0364] The above sample image of the incomplete shape may be a sample image taken of the actual state in which it is contained in a luggage bag, or it may be an image determined by dividing the above sample image of the complete shape into a plurality of sub-regions and selecting some of the divided sub-regions. The plurality of sub-regions may be, for example, rectangular areas defined by dividing the image at regular intervals, but are not limited thereto.

[0365] Dividing a sample image into multiple sub-regions can be referred to as patching.

[0366] The portion of the sample image selected corresponds to the area in which the sample image of the incomplete shape is actually accommodated in the luggage bag. In some embodiments, the portion of the sample image selected may be two or more adjacent areas. This is because, even if the luggage items accommodated in the luggage bag are partially obscured or wrinkled, certain parts of the body appear as a continuous shape of complete parts.

[0367] The second encoder (832) is configured to process the sample image of the complete shape among a pair of sample images to extract the features of the luggage item of the complete shape.

[0368] The first encoder (831) is configured to process the sample image of the incomplete shape among a pair of sample images to extract the features of the incomplete shape of the luggage item.

[0369] The output of the first encoder (831) may be a masked feature set from the feature set (e.g., a feature map or vector) of the second encoder (832). Features corresponding to sub-regions that are incompletely represented in the feature set of the second encoder (832) may be masked in a form in which they are removed. The output of the first encoder (831) includes position embeddings of sub-regions corresponding to the sample image of the incomplete shape input to the first encoder (831) among the partitioned sub-regions.

[0370] The decoder (835) may include a plurality of decoder blocks. The decoder (835) may be configured to implement an attention mechanism. The output of the second encoder (832) may be used as a key and a value in the attention mechanism, and the output of the first encoder (831) may be used as a query (q) in the attention mechanism.

[0371] Specifically, each decoder block may include a cross-attention layer and a self-attention layer. The output of the first encoder (831) pays attention to the output of the second encoder (832) through the cross-attention layer. Then, through the self-attention layer, the output of the first encoder (831) and the output of the second encoder (832) pay attention to each other.

[0372] The output sequence of the decoder (835) is a set of features available for predicting pixel values ​​for a complete shape sample image from an incomplete shape sample image. The decoder (835) is trained to predict a complete shape from an incomplete shape sample image and to restore features available for implementing the complete shape baggage item image. The trained decoder (835) can restore and output features that are identical or very similar to the features extracted from the complete shape sample image.

[0373] The features restored from the output sequence of the decoder (835) are features that can be extracted from the predicted complete shape of the luggage item image. When the output sequence of the decoder (835) is input into an image-generating AI model, the predicted complete shape of the luggage item image can be generated.

[0374] The output sequence of the decoder (835) is input to the fully connected layer (850).

[0375] The above-mentioned complete connection layer (850) can classify which of the preset products a luggage item appearing in an incomplete shape sample image corresponds to based on features restored by the decoder (835). The luggage item appearing in the incomplete shape sample image is recognized as a classified product.

[0376] The model components (810, 832, 831, 835, 850) of FIG. 10 above can be learned through various learning methods. For example, the model components (810, 832, 831, 835, 850) can be learned through self-supervised learning.

[0377]

[0378] In various embodiments of the present application, the second deep learning-based recognition model may be a foundation model pre-trained with a specific dataset that is fine-tuned to recognize incomplete or complete luggage items contained within a luggage bag. In some embodiments, after fine-tuning, the second deep learning-based recognition model may be further trained to recognize luggage items from images of incomplete luggage items.

[0379] Fine-tuning can be performed by adjusting the weights of the entire second deep learning-based recognition model or by adjusting a sub-network. The method of adjusting the sub-network can be performed by adjusting the conversion performance of the encoder's sub-layers (e.g., initial layer, end layer) into the low-dimensional feature space in the feature extraction layer (830) and / or at least some of the weights of the fully connected layer (850).

[0380] The above foundation model may have various ViT-based neural network architectures. For example, the above foundation model may be implemented as a Segment Anything Model (SAM), a Segment SAM, or a neural network architecture utilizing it.

[0381] In various embodiments of the present application, the foundation model may be pre-trained through the learning process of FIG. 10. In this case, the foundation model fine-tuned to be used as the second deep learning-based recognition model may include some or all of the components (810, 820, 832, 831, 835, 850) of FIG. 10.

[0382] For example, the foundation model may include an input layer (810), a second encoder (832), a first encoder (831), a decoder (835), a fully connected layer (850), and an output layer (870). A second deep learning-based recognition model fine-tuned from the foundation model may include an input layer (810), a first encoder (831), a decoder (835), a fully connected layer (850), and an output layer (870). The first encoder (831), decoder (835), and fully connected layer (850) of the foundation model and the first encoder (831), decoder (835), and fully connected layer (850) of the fine-tuned second deep learning-based recognition model may have different model parameters (e.g., weights) or set values.

[0383]

[0384] When a sub-image of an item area is input to the second deep learning-based recognition model trained in this manner, it outputs product information regarding the item appearing in the sub-image. The second deep learning-based recognition model, once trained, can accurately recognize what product category and what product a baggage item belongs to, regardless of whether it has an incomplete or complete shape.

[0385]

[0386] Referring again to FIG. 6, the user's terminal device (100) can search for weight data of luggage through a search engine or server (300) based on the recognition result (e.g., product information) of each luggage item recognized in the image inside the bag (S135).

[0387] Since the luggage weight search process of the above step (S135) is similar to the luggage bag weight search process of step (S115), a detailed explanation is omitted.

[0388] The weight information of the luggage is calculated as the total weight of the luggage by summing the weight of the luggage bag and the weight information of each recognized luggage item (S137).

[0389] In some embodiments, the electronic device (100) may be further configured to calculate the total weight of the luggage by taking into account the weight of the items contained within the collection item when the recognized luggage item is a preset collection item (S137). The collection item may be a pouch or another bag smaller than a luggage bag.

[0390] Specifically, the step of calculating the user's luggage data (S137) includes: displaying a UI screen that induces user input regarding the weight of luggage items inside a pouch contained in the internal space of the collection items when the recognized luggage items include a preset collection items; calculating the total weight of the collection items according to the input user input; and calculating the total weight information of the user's luggage based on the total weight of the collection items.

[0391] The user's terminal device (100) may display a GUI screen that prompts user input regarding product information of the luggage items inside the pouch. Input of information regarding the luggage items inside is required. The GUI screen may be configured to receive input for the product category (or product name) and quantity of the items included inside the pouch. The user input may be text input or click, touch, or other selection input.

[0392] The user's terminal device (100) can determine the luggage items inside the pouch as luggage items that are smaller than the size of the pouch and can be accommodated in the internal space, based on the product information of the luggage items inside the pouch and the size data of the pouch, and can search for weight information for each of the determined luggage items inside the pouch.

[0393] The user's terminal device (100) can calculate the total weight of the pouch by summing the weights of each item of luggage inside the pouch, and ultimately generate more accurate luggage item data.

[0394]

[0395] The total weight of the bundle is calculated by adding the weight of the bundle itself (e.g., the weight of the pouch) and the weight of the entered internal baggage items.

[0396] This weight information can be obtained for each recognized baggage item to calculate the user's baggage item data (S135). The baggage item data can be listed as recognition information and weight information for each individual item.

[0397] The user's terminal device (100) can generate luggage data including recognition information and weight information of the luggage bag, recognition information and weight information of the luggage items, and total weight information of the luggage (S137).

[0398] In some embodiments, the luggage data may further include a luggage list. The luggage list includes product information and corresponding weight information of the luggage bag, and product information and corresponding weight information of each luggage item.

[0399] The user's terminal device (100) can store at least one piece of information included in the luggage data, for example, product information and weight information of the luggage item, in the user's terminal device (100) (S137).

[0400] If multiple users board a single airport bus, the server (300) can obtain a set of luggage lists for multiple users who boarded the same airport bus. The set of luggage lists for the same airport bus is used to generate luggage lists for each airport bus.

[0401]

[0402] In alternative embodiments, the second deep learning-based recognition model may be configured and trained to recognize luggage items appearing in a sub-image without using size information of the luggage items. In this case, the step of generating the user's luggage item data (S130) includes: applying the user's bag interior image to a deep learning-based segmentation model to segment the item area where the luggage items appear in the user's bag interior image, thereby extracting a sub-image representing the item area (S131); applying the sub-image of the extracted item area to the second deep learning-based recognition model to recognize at least one luggage item appearing in the user's bag interior image (S133); obtaining weight data of the luggage items contained in the user's luggage bag through a search engine or server (300) based on the luggage item recognition result (S135); and generating luggage item data including the luggage item recognition result and weight information per luggage item (S137).

[0403]

[0404] Additionally, the step of calculating the user's baggage data (S100) includes: a step of checking whether the user's baggage data satisfies the pre-set airline's checked baggage weight limit (S150); a step of checking whether the user's baggage satisfies the pre-set checked baggage restriction guide based on the user's baggage data (S150); and a step of transmitting the confirmed user's baggage data to a server (300) (S170).

[0405] Checked baggage weight limits may vary by airline. For example, the checked baggage weight limit may be 15kg per user, but is not limited to this.

[0406] Additionally, checked baggage weight limits may vary depending on the user's payment information. Even with the same airline, some users may be allowed a higher baggage limit.

[0407] The user's electronic device (100) can set and store the baggage limit weight for the user based on payment information for the airline's mobile aircraft boarding pass.

[0408] FIG. 11 is a schematic diagram of a consignment restricted article guide according to various embodiments of the present application.

[0409] The user's terminal device (100) can store in advance a guide on restricted items that describes baggage that is restricted from being checked in and / or baggage that is allowed to be checked in.

[0410] Items restricted from checked baggage may be prohibited items and / or items with volume restrictions. Prohibited items may include, for example, tools, explosives, etc. Items with volume restrictions are baggage items with weight or volume limits per baggage, and may include, for example, liquids, batteries, medicines, etc.

[0411] The consignment restriction guide may include information on items prohibited from consignment and / or information on items with capacity limits, as illustrated in FIG. 11. The consignment restriction guide may consist of a list of items prohibited from consignment and a list of items with capacity limits.

[0412] A step (S150) of checking whether the user's baggage satisfies a pre-set check-in restriction guide based on the user's baggage data comprises: a step of checking whether the user's baggage corresponds to the check-in restriction weight; a step of providing a first warning message to the user containing pre-stored check-in restriction weight information for the user if the user's baggage is confirmed to be within the check-in restriction weight; a step of checking whether restricted items are included among the recognized baggage items; a step of providing a second warning message to the user containing the detection result of restricted items if it is confirmed that restricted items are included; and a step of providing a pass message to the user if the baggage does not correspond to the check-in restriction weight and no restricted items are confirmed. Additionally, the step (S150) may further include a step of updating the user's baggage data according to a user input indicating the removal of baggage items, which is entered after providing the first warning message or the second warning message.

[0413] The first warning message is a warning message indicating that the user's baggage weight exceeds a pre-stored capacity limit. The first warning message may include pre-stored information on items with capacity limits, information on the user's current baggage weight, and the capacity limit.

[0414] The second warning message is a warning message indicating that restricted checked baggage is included in the baggage. The second warning message may include an explanation of which baggage items are restricted items.

[0415] The user's terminal device (100) can search for luggage that corresponds to restricted checked luggage among the recognized luggage items, generate a list of restricted checked luggage consisting of the user's luggage found to correspond to restricted checked luggage, and generate a second warning message including the list of restricted checked luggage (S150).

[0416] The user's terminal device (100) can output a first warning message and a second warning message audibly or visually. The user's terminal device (100) can output the warning message when restricted checked baggage is included in the baggage bag. Through the warning message, the user can be prompted to remove baggage items corresponding to restricted checked baggage from the baggage contained in the bag.

[0417] When the user's terminal device (100) receives user input indicating the removal of the user's restricted checked baggage, it can update the existing user's baggage weight information by subtracting the baggage weight corresponding to the user input indicating the removal of the user's restricted checked baggage from the user's baggage weight calculated in step (S130) (S150).

[0418] The user's terminal device (100) can transmit the user's baggage data to the server (300) (S170). The user's terminal device (100) can transmit the baggage data along with the user's account information and / or mobile aircraft boarding pass data.

[0419] The server (300) can store the received user's luggage data (S190).

[0420] In some embodiments, the server (300) may store the user's luggage bag image and the bag interior image in association with the luggage data (S190).

[0421] Additionally, the server (300) can store the received baggage data by mapping it to the user's account and / or mobile aircraft boarding pass.

[0422] The server (300) can manage and store luggage data as usage history along with luggage bag images, luggage interior images, and mobile aircraft boarding passes (S170).

[0423]

[0424] Referring again to FIG. 5, the airport bus-linked checked baggage handling method includes: a step (S300) of verifying the identity of a user by analyzing a face image of a user boarding the airport bus, which is captured by a terminal device (200) of the airport bus installed in the airport bus to be used by the user to arrive at the airport to board the aircraft; a step (S400) of verifying the airport bus boarding pass of the user boarding the airport bus by the terminal device (200) of the airport bus; a step (S500) of authenticating the user's intention to board the aircraft based on the user's GPS data received from the user's terminal device (100) and the airport bus's GPS data by the terminal device (200) of the airport bus; and a step (S600) of requesting and obtaining the mobile aircraft boarding pass data of the user whose intention to board the aircraft has been authenticated from the user's terminal device (100) or server (300) at the terminal device (200) of the airport bus, and generating a baggage linkage code corresponding to the user's mobile aircraft boarding pass based on the user's baggage data to generate baggage tag information.

[0425] In the above step (S300), the terminal device (200) of the airport bus can generate a face image of a user boarding the airport bus by photographing the user's face through a camera (205), and can obtain the user's identity information by inputting the photographed face image of the user into a pre-trained identity verification model. The user's identity information may include a name, date of birth, phone number, address, profile image, and / or other profile information.

[0426] The above identity verification model can be learned using a profile photo of a user registered in the system (1), a passport photo provided by a public institution, or other profile photos.

[0427] As the deep learning-based identity verification model has been described above, a detailed explanation is omitted.

[0428] The terminal device (200) of the airport bus can access the server (300) and obtain account information having identity information that matches the user's identity information as the user's account information.

[0429] In various embodiments of the present application, the user's face information constitutes personal information. The system (1) is configured to use identity information by analyzing a face image only for users who have agreed to the use of personal information by inputting location information of the bus stop where the user will board the airport bus and information of the boarding time into the system. Specifically, the step of obtaining the user's identity information (S300) may include: a step of obtaining the user's identity information by analyzing the user's face image; a step of checking whether a user having the obtained identity information is included in a list of bus passengers obtained in advance from the server (300); and a step of finally obtaining the identity information only for users included in the list of bus passengers.

[0430] The user can input location information of the bus stop where they will board the airport bus and boarding time information into the system in advance through their terminal device (100).

[0431] The server (300) can generate a list of bus passengers consisting of stop and boarding time information for each user based on the user's stop and boarding time information, and provide it to the terminal device (200) of the airport bus.

[0432] The terminal device (200) of the airport bus captures a user's face image at a specific stop at a specific time and primarily analyzes the user's identity information from the captured user's face image.

[0433] The system (1) may be configured to use the analyzed identity information only when the user whose identity information has been analyzed belongs to the list of users scheduled to board at the specific stop and specific time in the bus passenger list. On the other hand, the system (1) may be configured not to use the analyzed identity information when the user whose identity information has been analyzed does not belong to the list of users scheduled to board at the specific stop and specific time in the bus passenger list.

[0434] Additionally, in some embodiments, the step of verifying the identity of a user (S300) may include: a step of outputting the fact of identity verification failure through an output device (208) if the identity of the user fails as a result of analyzing the user's face image; a step of capturing a second face image of the user through a UI screen capable of capturing the user's real-time face at the time of boarding in the terminal device (100) of the user whose identity verification failed; a step of obtaining the user's identity information by inputting the second face image in the user's terminal device (100) into a pre-trained identity verification model; and a step of transmitting the obtained user's identity information to a server (300) or a terminal device (200) of an airport bus.

[0435] The terminal device (200) of the airport bus can output the fact of identity verification failure visually or audibly.

[0436] The second face image may be an image taken from a different angle (e.g., the front angle of the terminal device (100)) than the face image taken at the time of boarding.

[0437] The above identity verification model may be the same as the model provided to the airport bus terminal device (200).

[0438] In some other embodiments, the step of verifying the user's identity (S300) may include: a step of outputting the fact of failure of identity verification through an output device (208) if the user's identity verification fails as a result of analyzing the user's face image; a step of obtaining the user's biometric information through a UI screen requesting the user's biometric information input at the user's terminal device (100); and a step of verifying the user's identity based on the user's biometric information. The biometric information may be fingerprints or face information. The user's terminal device (100) may include a sensor or camera (205) for obtaining fingerprints or face information.

[0439] The account information of the verified user is shared with the terminal device (200) and server (300) of the airport bus.

[0440] Through this process, more reliable facial recognition-based identity verification can be performed to more reliably verify the identity of the user requesting checked baggage. As a result, the authenticity of the user's GPS used in the following step (S500) can be better guaranteed.

[0441] The terminal device (200) of the airport bus can check whether the user who boarded the airport bus has an airport bus boarding ticket in advance (S400).

[0442] The terminal device (200) of the airport bus is configured to perform an operation to check for the presence or absence of an airport bus boarding pass through scanning and / or other means.

[0443] In some embodiments, the step of verifying the airport bus boarding pass (S400) may include: a step of verifying whether there is a mobile airport bus boarding pass associated with the user's account information when the user's account information is obtained by verifying the user's identity through facial recognition of the step (S300); a step of outputting a message indicating that there is a mobile airport bus boarding pass through an output device (208) if there is a mobile airport bus boarding pass associated with the user; a step of proceeding with payment for the mobile airport bus boarding pass in real time based on the user's payment method previously registered in the system if there is no mobile airport bus boarding pass associated with the user; and a step of outputting a message indicating that payment has been completed.

[0444] The terminal device (200) of the airport bus can capture a user's face image through a camera 205 within the device and identify a matching user account by comparing the captured face image with a face image stored in the user DB of the system (1). The terminal device (200) of the airport bus can verify payment method information registered in the server (300) of the system (1) associated with the identified user account.

[0445] The terminal device (200) of the airport bus can be configured to check whether the user has a mobile airport bus boarding pass without scanning. The terminal device (200) of the airport bus inquires with the server (300) about the existence of a mobile bus boarding pass associated with a specific user's account, and the server (300) checks whether a valid mobile bus boarding pass associated with the account exists and transmits the result to the terminal device (200).

[0446] If there is no valid mobile bus ticket, the terminal device (200) of the airport bus requests real-time payment from the server (300). The server (300) proceeds with the payment of the bus ticket using a payment method associated with a specific user's account. Specifically, if the payment method associated with the user's account is a credit card, the server (300) requests payment approval from the card company system, and if it is a simple payment service, it requests payment approval from the third-party PG company system. As described above, the user's terminal device (100) can pre-register the user's simple payment method with the server (300).

[0447] When payment approval is completed, the server (300) transmits payment completion information to the terminal device (200), and the terminal device (200) outputs the received payment completion information to the bus driver through the display (208).

[0448] In some other embodiments, the step of verifying the airport bus boarding pass (S400) may include the step of verifying whether the user boarding the airport bus possesses the mobile airport bus boarding pass by scanning a two-dimensional image corresponding to the mobile airport bus boarding pass displayed on the user's terminal device (100) by a scanner (206) and reading the mobile airport bus boarding pass data included in the two-dimensional image.

[0449] The user's terminal device (100) can pay for the user's mobile airport bus ticket in advance before boarding the airport bus, obtain mobile airport bus ticket data to be used by the user to arrive at the airport to board the aircraft, and store the data.

[0450] The above mobile airport bus boarding pass data is used to display two-dimensional images, such as barcodes and QR codes, on the user's terminal device (100).

[0451] The scanner (206) can scan a two-dimensional image displayed by the terminal device (100) of a user who has made a prepayment.

[0452] In some other embodiments, the terminal device (100) of the airport bus may be configured to perform both the operation of checking for the presence of a mobile airport bus boarding pass through scanning and the operation of checking for the presence of a mobile airport bus boarding pass through facial recognition.

[0453] In the above step (S500), the GPS coordinates of the terminal device (200) of the airport bus are used as the GPS coordinates of the airport bus.

[0454] The step of authenticating the user's intention to board an aircraft (S500) may include: a step of determining whether the airport bus has entered a pre-set designated section based on the airport bus's GPS data; a step of determining whether the user's GPS location and the airport bus's GPS location match at at least one point on the designated section when the airport bus has entered the pre-set designated section; and a step of acknowledging the user's intention to board an aircraft when it is determined that the user's GPS location and the airport bus's GPS location match at at least one point on the designated section.

[0455] In the above step (S500), the designated section may be a specific section of the entire route of the airport bus that includes the regular airport stop. The designated section corresponds to a route where arrival at the airport is certain without the possibility of deviating midway. In some embodiments, the designated section may be part or all of the section between the first airport stop and the non-airport stop immediately preceding the first airport stop in the one-way route toward the airport. For example, if the airport bus is heading toward Incheon Airport, the designated section may be part or all of the route of the airport bus on the Incheon Airport Expressway. Additionally, in some embodiments, the designated section may be a section that must be passed on the road toward the airport stop. For example, the designated section may be part or all of the Yeongjong Bridge.

[0456] In some embodiments, the terminal device (200) of the airport bus may initiate an operation (S500) to authenticate the user's intention to board the aircraft when the GPS coordinates of the airport bus enter a designated section.

[0457] The terminal device (200) of the above airport bus compares the GPS coordinates of the airport bus and the user's terminal 100 to determine if they are in the same location within a certain time interval, and can authenticate that the user of the terminal device (100) within a certain distance has the intention to board the aircraft (S500).

[0458] To perform the above step (S600), the user's terminal device (100) or server (300) may acquire the user's mobile aircraft boarding pass data. The user may purchase a mobile aircraft boarding pass for the aircraft they wish to board through their terminal device (100) and proceed with a pre-check-in process for the aircraft for which they have purchased the boarding pass, thereby purchasing the mobile aircraft boarding pass for the aircraft in advance before boarding the airport bus. As described above, the user's terminal device (100) may purchase the mobile aircraft boarding pass through a dedicated application of the airport bus-linked checked baggage handling system (1), or through an external website or external application. The pre-purchased mobile aircraft boarding pass data is stored in the server (300) associated with the user's account information, or is stored in the user's terminal device (100).

[0459] The terminal device 200 of the airport bus can request and obtain the user's mobile aircraft boarding pass data from the server (300) or the terminal device (100).

[0460] In the above step (S600), the baggage linkage code is a tracking code assigned to the system (1) to manage checked baggage. The baggage linkage code is an identification code different from the baggage identification code, i.e., the airline baggage identification code, which was previously issued by airlines when accepting checked baggage. The baggage linkage code is assigned to the baggage of a user utilizing the airport bus-linked checked baggage processing method of the system (1). Among the checked baggage loaded onto an aircraft, the baggage linkage code is not assigned to the checked baggage of a user who does not use the system (1). On the other hand, among the checked baggage loaded onto an aircraft, the baggage linkage code and the airline baggage code are assigned to the checked baggage of a user who uses the system (1).

[0461] If a single user wishes to check in multiple luggage bags, a different luggage linkage code is assigned to each bag.

[0462] In some embodiments, the terminal device (200) of the airport bus may generate a baggage linkage code based on the identification information of the airport bus, the identity information of the user, and baggage sequence number information assigned to the baggage bag. The baggage sequence number information may be the sequence in which the baggage was loaded onto the airport bus, or the sequence in which the user boarded the airport bus. The terminal device (200) of the airport bus may generate a baggage linkage code by combining these information. Through this, the server (300) can identify the airport bus that delivered the baggage of the user to whom the baggage linkage code was assigned.

[0463] The server (300) can receive and store the user's identity verification result confirmed in step (S300) and the user's corresponding baggage linkage code(s) generated in step (S600) from the terminal device (200) of the airport bus.

[0464] When the server (300) receives a baggage linkage code, the user's baggage and baggage bag are received by the system (1) as checked baggage and are determined as targets for the checked baggage processing service according to the embodiments of the present application.

[0465] In addition, in various embodiments of the present application, the terminal device (200) of the airport bus can communicate with an airline system to request and obtain an airline baggage identification code for each piece of luggage belonging to a user loaded on the airport bus. Then, the system (1) can generate new baggage tag information in which a baggage linkage code is added to a conventional baggage tag.

[0466] FIG. 13 is a detailed flowchart of the process of generating baggage tag information according to various embodiments of the present application.

[0467] Referring to FIG. 13, the step of generating a baggage linkage code to generate baggage tag information (S600) may include: requesting and obtaining mobile aircraft boarding pass data for the aircraft of the user whose intention to board the aircraft is authenticated by the terminal device (200) of the airport bus from the user's terminal device, and generating a baggage linkage code corresponding to the user's mobile aircraft boarding pass based on the baggage item data of the user who boarded the airport bus (S610); when the server (300) obtains the user's baggage linkage code, associating the user's baggage linkage code with the user's account information and the user's aircraft boarding pass information (S630); generating baggage tag information for each of the user's bags including the user's baggage linkage code at the server (300) (S650); and transmitting a linkage service reception message including the user's baggage tag information to the user's terminal device (100) at the server (300) (S670).

[0468] When the terminal device (200) of the airport bus generates a baggage linkage code, it can transmit the user's identity verification result and the baggage linkage code to the server (300) through the communication unit (201) (S610). Then, the server (300) can associate the user's baggage linkage code with the user's account information and the user's aircraft boarding pass information, and store and manage it as a usage history.

[0469] Aircraft boarding pass information is information included in mobile aircraft boarding pass data, and may include, for example, aircraft information, flight schedule information (boarding time, estimated time of arrival, and / or estimated flight time, etc.), passport information (name, date of birth, etc.).

[0470] Through this association operation, the baggage linkage code is mapped to the user's account, aircraft boarding pass, and passport information on the system (1).

[0471] Baggage tag information includes information included in conventional baggage tags and a baggage linkage code. The information included in conventional baggage tags includes user information (i.e., passenger information), airline information, destination information, and an airline baggage identification code represented by a tracking barcode or QR code. The passenger information is the name of the boarding pass holder and reservation information. The airline information included in conventional baggage tags is the code of the airline handling the baggage check-in. However, the airline information within the baggage tag information provided by the present system (1) is the unique code of the airline the passenger first boards.

[0472] The server (300) can request an airline baggage identification code from the airline system and receive an airline baggage identification code for each user's baggage. The server (300) can generate a baggage tag including the baggage linkage code generated by the terminal device (200) of the airport bus and the assigned airline baggage identification code (S630).

[0473] The generated baggage tag information can be printed and attached to the user's baggage. The printed baggage tag includes a baggage linkage code.

[0474] The printed material of the above luggage tag can be attached to a location that appears clearly in the second luggage bag image in step (S800) described below.

[0475] The terminal device (200) of the airport bus can transmit a list of baggage linkage codes generated for users who have boarded the airport bus to the server (300). This list may be a list of baggage linkage codes and user identity verification results (or account information).

[0476] The server (300) can identify the user's terminal device (100) based on the user's identity verification result and transmit a linkage service acceptance message including the user's baggage tag information to the user's terminal device (100). The server (300) can transmit a linkage service acceptance message including the corresponding baggage linkage code to the user's terminal device (100) corresponding to the user's identity verification result confirmed in step (S300) (S670).

[0477] FIG. 12 is a schematic diagram of a baggage linkage service reception message according to various embodiments of the present application.

[0478] Referring to FIG. 12, the linkage service reception message may include a baggage linkage code. In addition, the linkage service reception message may further include an airline baggage identification code, etc., along with the baggage linkage code.

[0479] In some embodiments, the linkage service reception message may further include baggage data. Then, the terminal device (100) of the user who received the linkage service reception message may display a message that further includes item information recognized as baggage and baggage bag information.

[0480] The above airport bus terminal device (200) can perform the operation of steps (S300 to S600) whenever a user boards the bus at a bus stop.

[0481] In various embodiments of the present application, the airport bus-linked checked baggage processing method may further include: a step in which a server (300) compares a list of bus passengers of verified users with a list of users scheduled to board at a specific stop and time stored in the server (300) to determine if there are any users among the scheduled users who have not boarded the airport bus, and if it is confirmed that there are users who have not boarded, transmits a non-boarding inquiry message to the terminal device (100) of the user who has not boarded based on the ID of the user who has not boarded; a step in which, if a boarding fact correction message is received from the terminal device (100) of the user who received the non-boarding inquiry message, an update of the bus passenger list for the airport bus based on the ID of the user; a step in which a request to generate a baggage linkage number is transmitted to the terminal device (200) of the airport bus for the user who transmitted the boarding fact correction message; and a step in which the baggage linkage number generated for the user who transmitted the boarding fact correction message is received and the updated bus passenger list is verified.

[0482] The user's terminal device (100) that receives the above non-boarding inquiry message can transmit a message to the server (300) to modify the boarding fact. The terminal device (100) can display the content of the non-boarding inquiry message and a button (e.g., "actually boarded") to generate a message to modify the boarding fact.

[0483] The user's terminal device (100) can capture the user's face image in real time and generate the user's identity verification result through an identity verification model included in the terminal device (100), and transmit a boarding fact correction message including the user's real-time face image and the corresponding identity verification result to the server (300). The identity verification result may have the same data item as the identity verification result obtained in the identity verification step (S300).

[0484] In some embodiments, the user's terminal device (100) may generate a boarding fact correction message including the user's real-time face image, the user's ID, and an identity verification result (e.g., profile information) and transmit it to the server (300).

[0485] The above scheduled time may be a time interval set from the time of selection of the button or the time of capture of the real-time face image.

[0486] The above baggage linkage number generation request may include information used to generate a baggage linkage number. The above baggage linkage number generation request may include a user ID, and information about a device or network to which the terminal device (200) can connect a communication channel with respect to the terminal device (100).

[0487] The terminal device (200) of the airport bus above skips the identity verification step (S300) for the user who receives the baggage linkage number generation request, and generates a baggage linkage number (S600) by performing steps (S400 to S500) based on the generation request.

[0488] Even if the user sends a false message correcting their boarding status, there is no problem. This is because if the user who sent the message correcting the boarding status does not actually board the airport bus, the intention to board the aircraft according to step (S500) is calculated as an authentication failure.

[0489] Then, baggage linkage numbers are generated for all verified users or users missing due to system errors, and information about them is shared and managed on the employee terminal device (600) and server (300).

[0490] Referring again to FIG. 5, the airport bus-linked checked baggage processing method includes the step (S700) of transmitting baggage tag information to the employee terminal device (600) in order to print baggage tag information including a baggage linkage code and an airline baggage identification code from the server (300) - the employee terminal device (600) is a terminal device of an employee responsible for the process from when the airport bus arrives at the airport to the baggage inspection system (800) inside the airport during the airport bus-linked checked baggage processing process - ;

[0491] The employee terminal device (600) is a terminal device for an employee that transmits or manages the users' checked baggage loaded on the airport bus to the baggage inspection system (800) inside the airport after the airport bus arrives at the airport.

[0492] The server (300) transmits a list of luggage loaded on the airport bus to an employee terminal device (600) (S700). The list of luggage by airport bus is a list consisting of user account information and luggage tag information. When multiple users board the airport bus, the list of luggage by airport bus consists of luggage tag information for the luggage bags of multiple users.

[0493] The employee terminal device (600) can print a label sticker containing baggage tag information from the baggage list through an internally mounted print unit or an externally connected printer via wired or wireless connection. The label sticker is attached to the user's checked baggage (S700).

[0494] The label sticker may include a two-dimensional image containing luggage tag information. The two-dimensional image may be represented as a barcode or QR code.

[0495] In various embodiments of the present application, the step (S700) of transmitting the baggage tag information to the employee terminal device (600) may involve transmitting a baggage bag image corresponding to a baggage linkage code within the baggage tag information. Then, the employee can know which baggage bag has a label sticker with which baggage tag information to attach, thereby preventing incorrect attachment of the label sticker.

[0496] In some embodiments, the employee terminal device (600) may be further configured to provide an attachment guide indicating the attachment location of the label sticker of the luggage tag on the luggage bag. Then, in the following step (S800), the luggage inspection system (800) can recognize the luggage tag more accurately.

[0497] If multiple users board a single airport bus, the employee terminal device (600) can print a set of label stickers displaying luggage tag information for each of the luggage bags of the multiple users who boarded the same airport bus. The set of label stickers is attached to the corresponding luggage bags.

[0498] The above-mentioned employee terminal device (600) can display a list of luggage for each airport bus. Then, in the above-mentioned airport bus-linked checked baggage handling system (1), an employee located at the boarding airport can safely manage luggage bags assigned during the route from the airport bus to the luggage inspection system (800) by comparing the attached label sticker with the luggage tag information within the list of luggage for each airport bus displayed on the employee terminal device (600).

[0499] In addition, the airport bus-linked checked baggage handling system (1) may be configured to additionally check whether the baggage bags are subject to checked baggage handling that must be managed during the boarding stage through an RFID tag (700).

[0500] Specifically, the airport bus-linked checked baggage processing method comprises: a step (S710) of transmitting the user's baggage tag information to the employee terminal device connected to the RFID reader module in order to store the user's baggage tag information in the initialized RFID tag at the server (300); a step (S720) of storing the user's baggage tag information received through the RFID reader module in the initialized RFID tag (700) and transmitting the UID of the initialized RFID tag (700) to the server (300); and a step (S730) of receiving the user's baggage tag information and the UID of the RFID tag transmitted from the RFID reader module to the RFID tag at the server (300), associating them with each other, and updating the usage history of the user having the baggage tag information associated with the UID of the RFID tag. Additionally, in some embodiments, the airport bus-linked checked baggage handling method may further include the step (S750) of obtaining tag information stored in the RFID tag (700) attached to the user's baggage bag (400) from the employee terminal device (600).

[0501] The server can identify an employee terminal device (600) to which baggage tag information will be sent based on the user's airport bus schedule, the employee's work schedule, and / or the GPS location of the employee's terminal device (600). The server transmits the user's baggage tag information to the identified employee terminal device (600), and when the initialized RFID tag attached to the user's baggage bag arrives at the airport bus stop and is scanned by a scanner (700) connected to the employee terminal device (600), the baggage tag information can be stored in the initialized RFID tag.

[0502] The above airport bus-linked baggage handling system (1) is configured to implement a baggage handling service at the boarding stage using an RFID tag (700).

[0503] As described above in FIG. 5 and step (S10), the initialized RFID tag (700) includes a memory (703) which is a storage capable of storing the user's luggage tag information, and a communication unit (701) that exchanges information with a reader module, and the initialization state represents a state before storing the UID and before any tag information is stored.

[0504] The server (300) transmits the user's baggage tag information generated in the above step (S700) to the employee terminal device (600), and the employee terminal device (600) can transmit the user's baggage tag information to the initialized RFID tag (700) through an RFID reader module that communicates with each other (S710). The baggage tag information of multiple users can be transmitted to each of the multiple initialized RFID tags (700).

[0505] The initialized RFID tag (700) can safely store the user's luggage tag information received through the RFID reader module (S720). As a result, various information identifying the luggage bag, such as the luggage linkage code and the airline's luggage identification code, is implanted into the RFID tag (700).

[0506] In various embodiments of the present application, the step (S730) of storing user baggage tag information received through an RFID reader module in the initialized RFID tag (700) may include: the step of encrypting the user baggage tag information received through an RFID reader module by the initialized RFID tag (700); and the step of storing the encrypted user baggage tag information in a memory (703).

[0507] The user's baggage tag information can be encrypted by various encryption methods. In some embodiments, the processor (702) can encrypt the user's baggage tag information using AES, RSA, or ECC methods.

[0508] Specifically, the AES (Advanced Encryption Standard) is a powerful symmetric-key encryption algorithm that guarantees data confidentiality and has the advantage of being relatively simple to implement. The AES method encrypts data in blocks using a fixed key length (128, 192, or 256 bits), providing fast processing speed and high security.

[0509] RSA (Rivest-Shamir-Adleman) is an asymmetric key encryption algorithm that uses two keys, a public key and a private key, to simultaneously guarantee the confidentiality and integrity of data. The RSA method is primarily used for data encryption as well as digital signatures, and is characterized by easy key management and high security.

[0510] ECC (Elliptic Curve Cryptography) is an asymmetric key encryption algorithm based on elliptic curve theory. It has the advantage of providing the same level of security as RSA with shorter key lengths. Due to its high computational efficiency, the ECC method is suitable for devices with limited resources, such as RFID tags.

[0511] In this way, various encryption methods applied to the RFID tag (700) can safely protect the user's luggage tag information based on their respective characteristics and advantages, and contribute to improving the overall security of the system.

[0512] When the RFID tag (700) stores the user's baggage tag information as new tag information, it transmits a storage completion message including its own UID to the employee terminal device (600) via the RFID reader module, and the employee terminal device (600) transmits the storage completion message and the corresponding user's baggage tag information to the server (300) (S730). When the server (300) receives the user's baggage tag information and the UID of the RFID tag (700) from the employee terminal device (600), it associates these pieces of information with each other (S730). The fact of association between the UID and the user's baggage tag information is updated in the user's usage history (S730). When the UID of the RFID tag (700) is added to the user's usage history, the fact of association between them is confirmed.

[0513] In some embodiments, the employee terminal device (600) may be further configured to provide an attachment guide indicating the attachment location of the RFID tag (700) on the label sticker of the luggage tag in the luggage bag.

[0514]

[0515] The employee terminal device (600) can obtain the user's luggage tag information stored in the RFID tag (700) attached to the luggage bag through an RFID reader module (S750). If the read tag information is the user's luggage tag information stored in step (S730), the employee can verify the user's luggage tag information (e.g., luggage linkage code, etc.) stored in the tag (700).

[0516] Additionally, the RFID reader module can obtain the user's baggage tag information by reading the user's baggage tag information stored in an encrypted state on the RFID tag (700) as tag information corresponding to the UID (S750).

[0517] If the previously stored tag information is encrypted, the employee terminal device (600) can obtain the tag information by decrypting the encrypted tag information (S750). Specifically, the step (S750) of obtaining the tag information stored in the RFID tag (700) attached to the user's luggage bag (400) from the employee terminal device (600) may include: the step of the RFID reader module reading the UID of the RFID tag (700); the step of reading the encrypted user's luggage tag information stored in the memory (703) of the RFID tag (700); the step of decrypting the read encrypted user's luggage tag information - the decryption is performed through a decryption method corresponding to the method (AES, RSA, or ECC) used for encryption of the tag information -; and the step of transmitting the decrypted user's luggage tag information to the employee terminal device (600).

[0518] At this time, the employee terminal device (600) displays the received user's baggage tag information on the screen so that the employee can check information such as the baggage linkage number.

[0519] In addition, the employee terminal device (600) can transmit the received user's baggage tag information to the server (300) to verify the validity of the information.

[0520] In various embodiments of the present application, the step (S750) may further include a step of verifying the acquired tag information. The process of verifying the acquired tag information in the step (S750) may be performed using a TOTP (Time-based One-Time Password) method.

[0521] TOTP is a two-factor authentication (2FA) method that uses a time-based algorithm to generate a unique one-time password that changes at regular intervals. TOTP generates a one-time code based on a secret key and the current time value, and the generated code is valid only for a specified time (typically 30 seconds).

[0522] FIG. 14 is a detailed flowchart of a process for verifying baggage tag information stored in an RFID tag according to various embodiments of the present application.

[0523] Referring to FIG. 14, the step of verifying the acquired tag information includes: a step (S751) in which the employee terminal device (600) transmits a wireless signal requesting the creation of a TOTP to the RFID tag (700) through the RFID reader module; a step (S752) in which the processor (702) of the RFID tag (700) creates a TOTP and transmits it as a response signal; a step (S753) in which the TOTP of the response signal is received through the RFID reader module; a step (S754) in which the employee terminal device (600) transmits the transmitted TOTP to the server (300) and the server (300) verifies the validity of the TOTP; a step (S755) in which, if the TOTP is valid, the server (300) transmits a verification success message to the employee terminal device (600); and a step (S756) in which, if the result of the validity verification of the TOTP is abnormal, the user terminal device (100) associated with the RFID tag (700) transmits an RFID status notification message.

[0524] In the above system (1), the following preparation process can already be performed to perform TOTP verification through the RFID tag (700):

[0525] a) A step in which a processor (702) installed in an RFID tag (700) generates and stores a unique secret key;

[0526] b) A step of maintaining the accurate time required for TOTP generation by setting time synchronization between the RFID tag (700) and the server (300).

[0527] Prior to the above step (S751), each RFID tag (700) generates a unique secret key. This secret key serves as the basis for TOTP generation and is generated using a secure random number generator for security purposes. The generated secret key is stored in a secure memory area of ​​the RFID tag (700) and is protected from external access.

[0528] In the above step (S751), Network Time Protocol (NTP) or a similar time synchronization protocol is used to ensure accurate time synchronization between the RFID tag (700) and the server (300). This is to accurately utilize the time-based characteristics of TOTP. The RFID reader module and the server (300) perform timestamp management operations and continuously synchronize time to minimize time deviations during TOTP generation and verification.

[0529] This time synchronization is implemented based on the unique secret key of each RFID tag (700), etc.

[0530] In the above step (S751), the RFID reader module transmits a wireless signal containing a specific command requesting the RFID tag (700) to generate TOTP.

[0531] In the above step (S752), the RFID reader module securely transmits the received TOTP to the employee terminal device (600). At this time, encryption is applied during the data transmission process. In some embodiments, the RFID reader module may encrypt the TOTP using Transport Layer Security (TLS) or other security protocols for secure data transmission.

[0532] In the above step (S752), the processor (702) of the RFID tag (700) receives a secret key and the current time as input and generates a one-time password using a TOTP algorithm (e.g., HMAC-based TOTP defined in RFC 6238). The generated TOTP is transmitted as a response signal to the RFID reader module, and additional encryption or signatures may be applied during this process to ensure the integrity of the data.

[0533] In the above step (S753), the RFID reader module securely transmits the received TOTP to the employee terminal device (600). At this time, encryption is applied during the data transmission process, and TLS (Transport Layer Security) or other security protocols are used for secure data transmission.

[0534] In the above step (S754), the server (300) verifies the validity of the received TOTP by comparing it with the secret key of the corresponding RFID tag (700) and the TOTP generated based on the current time. When verifying the TOTP, a time window may be set to allow for a slight time deviation (e.g., current time ±1 minute).

[0535] If the TOTP is valid in the above step (S754), the employee terminal device (600) may display verified tag information to compare with the luggage tag information shown on the label sticker attached to the luggage bag (S755). Then, the reliability of the employee's verification result is further ensured.

[0536] Additionally, if TOTP is valid, the server (300) searches for the user associated with the RFID tag (700) and sends a baggage status notification message to the user's terminal device (100) describing the status of the user's checked baggage being processed normally (S755).

[0537] On the other hand, if TOTP is invalid, the server (300) searches for the user associated with the RFID tag (700) and sends a baggage status notification message describing the abnormal state to the user's terminal device (100) (S756).

[0538] The RFID status notification message indicates an abnormal state in which the RFID tag (700) attached to the luggage bag at the time of validation and the RFID tag (700) stored as attached to the server (300) do not match each other. This message enables the user to quickly recognize and respond to the abnormal state of the luggage.

[0539] The server (300) can update the user's baggage tag information based on the verification result. Specifically, if the TOTP is valid, the server (300) updates the user's baggage tag information to reflect the location and status of the baggage in real time. The real-time location of the baggage is updated to the location of the employee terminal device (600).

[0540] Additionally, the airport bus-linked checked baggage processing method comprises: a step (S800) in which, when a server (300) receives a security inspection result including a baggage bag image taken of the user’s baggage bag and a reading result data obtained from inspection source data for the baggage bag from an airport baggage inspection system (800), the server associates the reading result data for the baggage bag with the user’s baggage linkage code based on the matching result between the second baggage bag image of the baggage inspection system (800) and the user’s baggage bag image, and the matching result between the reading result data of the baggage inspection system (800) and the user’s baggage item data; and a step (S900) in which a baggage check-in processing notification message is transmitted to the user’s terminal device (100) based on the inspection result message for the user’s baggage received from the baggage inspection system (800).

[0541] The above security inspection result may include a reading result including whether the user's luggage bag contains restricted checked baggage and / or an identification result of restricted checked baggage presumed to be in the luggage bag, as well as an inspection source for the luggage bag that serves as the basis for the reading result. If the security inspection device is an X-ray scanner, the security inspection result may include an X-ray image of the user's luggage bag and a reading result of the X-ray image.

[0542] The airport baggage inspection system (800) applies the users' checked baggage to a security inspection device to obtain inspection source data for the users' baggage bag, and provides a reading result based on the inspection source data to the server (300) as reading result data; and can provide a second baggage bag image taken of the baggage bag in the received state to the server (300).

[0543] For example, if the security inspection device is an X-RAY scanner, the airport baggage inspection system (800) may acquire an X-RAY image of a baggage bag as an inspection source and provide the result of reading it to a server (300). In this case, the reading result data obtained from the inspection source data for the baggage bag includes an X-RAY-based recognition result for at least one baggage item contained in the user's baggage bag, whether the baggage item corresponds to restricted checked baggage, and the type of the corresponding restricted checked baggage. For example, the result of reading the security inspection image of the baggage bag of FIG. 6 may include item information (e.g., item type) for each of at least some of the baggage items of FIG. 6.

[0544] If multiple baggage items correspond to restricted checked baggage, the server (300) can receive a list of restricted checked baggage as a result of reading the security inspection image (S800).

[0545] Additionally, the baggage inspection system (800) may provide a second baggage bag image to the server (300), which is a photograph of the baggage bag in its received state before / after the user's checked baggage received via the airport bus is applied to the security inspection device (S800). The second baggage bag image is an image of the exterior of the baggage bag captured in the visible light wavelength range. This second baggage bag image is an image taken at a different time than the same baggage bag image taken in step (S100). In some embodiments, the baggage inspection system (800) may acquire the second baggage bag image through a CCTV or other camera that photographs the bag being fed into the reader on a conveyor belt connected to the security inspection device.

[0546] The second luggage bag image is provided to the server (300) along with the reading result of the corresponding security inspection image (S800).

[0547] The above step (S800) may include: a step of determining whether the luggage bags appearing in each bag image match each other by comparing and analyzing the second luggage bag image of the luggage inspection system (800) and the user's luggage bag image received in the above step (S100) by the server (300); and a step of obtaining the user's luggage item data based on the luggage tag information appearing in the second luggage bag image, and calculating the matching result between the reading result data of the luggage inspection system (800) and the user's luggage item data by comparing and analyzing the reading result data of the luggage inspection system (800) and the obtained user's luggage item data.

[0548] The server (300) compares and analyzes the second luggage bag image of the luggage inspection system (800) and the user's luggage bag image received in step (S100) to determine whether the luggage bags appearing in each bag image match each other. The matching result can be calculated as a matching probability. To this end, the server (300) can use various image analysis methods to analyze whether different bag images represent the same bag.

[0549] In the above step (S800), the server (300) can determine whether the item information recognized from the security inspection image and the item information recognized from the luggage data calculated in step (S100) match each other in the result of reading the luggage inside the luggage bag received from the luggage inspection system 300. The matching result can be calculated as a matching probability. In some embodiments, the server (300) can calculate the matching result as a matching probability between types of luggage items (e.g., product groups) (S800).

[0550] The server (300) can extract luggage linkage codes and user information from luggage tag information appearing in the second luggage bag image, and search for user luggage item data that matches the extracted luggage linkage codes and user accounts. To do this, the server (300) is configured to use OCR or other text detection technology.

[0551] In some embodiments, if the server (300) fails to detect a luggage tag code in the second luggage bag image, it may search for the luggage linkage code of the user with the highest combination of the luggage bag matching result and the luggage item matching result, and associate the searched luggage linkage code with the luggage shown in the security inspection image and the reading result therefrom (S800).

[0552] This matching verification process is a double-check procedure that compares the external image of checked baggage with the attached label via the airport security system; by requiring additional verification from the user in the event of a bag discrepancy, it further ensures the reliability of the checked baggage handling service.

[0553] The inspection result message indicates that the baggage has passed through the airport security checkpoint and is allowed to be loaded onto an aircraft according to the security inspection of the baggage inspection system (800). That is, the system (1) utilizes both the self-inspection of step (S150) and the security inspection inside the airport.

[0554] The above inspection result message may be a rejection message indicating that restricted checked baggage was detected during the security inspection, or a pass message indicating that restricted checked baggage was not detected during the baggage security inspection.

[0555] Passage messages and rejection messages may include baggage tag information attached to the luggage bag. In other words, both pass and rejection messages include a baggage linkage code.

[0556]

[0557] FIG. 15 is a detailed flowchart of a process for transmitting a baggage check-in notification message according to various embodiments of the present application.

[0558] Referring to FIG. 15, the step of transmitting a baggage check-in notification message to the user's terminal device (100) (S900) may include: a step of transmitting a first notification message to the user's terminal device (100) based on a baggage linkage code in the baggage check message when a rejection message is received from the baggage inspection system (800) (S910); and a step of transmitting a second notification message including a baggage linkage code to the user's terminal device when a pass message is received from the baggage inspection system (800) based on a baggage linkage code in the pass message (S920).

[0559] Additionally, in some embodiments, the step (S900) of transmitting the baggage check-in notification message may further include the step (S930) of transmitting a third notification message, including a baggage verification location, to the user's terminal device (100) when the second baggage bag image and the baggage bag image do not match each other.

[0560] The baggage check-in notification message is a notification message explaining whether baggage has been checked in. The baggage check-in notification message may be provided as a first notification message or a second notification message depending on the results of the security inspection.

[0561] In the security inspection results, the rejection message indicates that restricted checked baggage among the user's baggage items was detected during the baggage security inspection, and that the user's baggage failed to pass the security inspection. The baggage inspection system (800) obtains the user's baggage linkage code based on the baggage tag attached to the user's baggage bag, and transmits a rejection message including the baggage linkage code and the rejection details as an inspection result to the server (300).

[0562] The first notification message may include at least one piece of information among baggage check location and baggage tag information, such as the baggage linkage code. The server (300) may search for a user's terminal device (100) corresponding to the baggage linkage code in the drop message, and transmit a first notification message including the baggage check location and the baggage linkage code to the searched user's terminal device (100) (S910).

[0563] The first alarm message indicates the location of the baggage check area. Additionally, the first alarm message indicates the fact that the baggage was missed during inspection. As a result, the first alarm message can call a user assigned a specific baggage linkage code to the baggage check area.

[0564] In the security inspection result, the pass message indicates that the user's baggage has passed the security inspection because no restricted checked baggage was detected during the baggage security inspection. The baggage inspection system (800) obtains the user's baggage linkage code based on the baggage tag attached to the user's baggage bag, and transmits a pass message including the baggage linkage code and the pass details as an inspection result to the server (300).

[0565] The second notification message above indicates baggage tag information and the fact of passing the security check. The second notification message can notify the user of the fact of passing the security check (S920).

[0566] The third notification message indicates that the user's luggage bag was not delivered to the airport baggage inspection system (800), or that the user's luggage bag and a different luggage bag were delivered to the baggage inspection system (800). Then, the system (1) can send a third notification message to the user's terminal device (100) indicating that there is an abnormality in the double check result and that the user should visit a specific storage location to check the bag.

[0567] The server (300) may send a third notification message if the comparison result between the second luggage bag image and the luggage bag image yields a matching probability less than a preset threshold probability. The threshold probability is a probability set to ensure that the bags appearing in the two images are the same bag.

[0568] Meanwhile, in the system (1), even if there is no actual addition or removal of luggage from the same bag, there is a possibility that the luggage items recognized from the security inspection image and the luggage items recognized from the luggage bag image may not match 100%. Even if the luggage items do not match 100% according to the image analysis results, no separate warning message is sent.

[0569] According to this airport bus baggage check-in system and method, passengers can minimize the burden of the time required to complete check-in and baggage procedures before boarding time. Ultimately, passengers can use the aircraft quickly and conveniently.

[0570] In addition, by allowing passengers to easily use public transportation such as airport buses instead of private vehicles, energy can be saved and the competitiveness of public transportation compared to other modes of transportation to reach the airport can be secured.

[0571] Meanwhile, the airport includes a check-in desk area including a passenger waiting area, a security area, a CIQ (customs, immigration, quarantine) area, a duty-free shop area, a restaurant area, a rest area, a boarding waiting area, and other internal airport spaces. If a user skips visiting the check-in desk due to the airport bus-linked baggage handling system (1) mentioned above, the check-in desk area can ultimately be replaced with another internal airport space, so that the airport operator and designer can utilize the internal space more efficiently for the boarding of aircraft users.

[0572]

[0573] Checked baggage handling services provided at the arrival stage

[0574] Additionally, referring to FIG. 1b, the airport bus-linked checked baggage handling system (1) is configured to provide checked baggage handling services to provide convenience to the user even in the stage after landing (hereinafter, arrival stage), such as the process of the user getting off the aircraft and passing through the arrival airport, or the process of getting off and passing through another airport in the same country.

[0575] To this end, in various embodiments of the present application, the airport bus-linked checked baggage handling system (1) may be further configured to include an employee terminal device (1600) in charge of checked baggage at an airport (hereinafter, arrival airport) in the area where the user disembarks from the aircraft (hereinafter, arrival area), an employee terminal device (1200) of a baggage bag transport vehicle, and an employee terminal device (1210) of a user transport vehicle that transports the user.

[0576] A user transport vehicle is a vehicle used by a user to travel between their accommodation and another location. The aforementioned user transport vehicle may be a tour bus, airport bus, van, multi-passenger taxi, or other buses or vehicles.

[0577] For clarity of the following description, the present application is described in more detail with examples in which the user's vehicle is a tour bus.

[0578]

[0579] The server (300) is configured to communicate with these components (1200, 1210, 1600). The transport vehicle for the luggage bag may be a bus, a truck, or various other means of transport. The employee terminal device (1200) of the transport vehicle may be a terminal device of a driver operating the transport vehicle or an employee in charge of the luggage bag loaded onto the transport vehicle.

[0580] A tour bus is a tour bus boarded by users of the checked baggage service who have disembarked from an aircraft. The initial destination of the aforementioned tour bus may be a location different from the initial destination of the transport vehicle.

[0581] The employee terminal device (1210) of the tour bus may be a terminal device of the driver of the tour bus or an employee in charge of passengers on the tour bus.

[0582] The employee terminal device (1600) is a terminal device for an employee participating in providing checked baggage services at the arrival airport, and corresponds to the employee terminal device (600) participating in providing checked baggage services at the boarding airport.

[0583] These terminal devices (1200, 1210, 1600) may be implemented as devices identical or similar to the employee terminal device (600). Specifically, the employee terminal device (1600) and the employee terminal device (1200) of the transport vehicle may include an RFID reader module, similar to the employee terminal device (600). Accordingly, these terminal devices (1200, 1600) can read the RFID tag (700) attached to the luggage bag at the arrival airport or before loading the luggage bag onto the transport vehicle, and obtain and display the user's luggage tag information stored in the RFID tag (700). Then, the employee or the employee of the transport vehicle can compare the displayed luggage tag information with the luggage tag information on the label sticker attached to the luggage bag to verify it.

[0584] For example, the employee terminal device (1600) may receive a list of baggage assigned to it from the server (300). The baggage list may be a list consisting of related user profile information, baggage bag images, and baggage tag information. In some embodiments, the baggage list may be generated per aircraft, and the server (300) may assign a baggage list for a specific aircraft to the employee terminal device (1600).

[0585] The above-mentioned employee terminal device (1600) can display the assigned baggage list. Then, in the above-mentioned airport bus-linked checked baggage handling system (1), an employee located at the arrival airport can safely manage the baggage bags assigned at the boarding airport by comparing the attached label sticker with the baggage tag information within the baggage list for each airport bus displayed on the employee terminal device (1600).

[0586] Additionally, the airport bus-linked checked baggage handling system (1) may be configured to further verify, through an RFID tag (700), whether the baggage bags are subject to checked baggage handling that must be managed during the boarding stage. Specifically, the employee terminal device (1600) can obtain the user's previously stored baggage tag information by reading the tag information stored in the RFID tag (700) attached to the baggage bag through an RFID reader module. If the read tag information is the user's baggage tag information stored in step (S730), the employee can verify the user's baggage tag information (e.g., baggage linkage code, etc.) stored in the tag (700). Since the process of verifying this RFID tag information has been described in detail in step (S750), a detailed explanation is omitted.

[0587] The operation of these components (1200, 1210, 1600) is described in more detail below.

[0588]

[0589] Additionally, the airport bus-linked checked baggage handling method illustrated in FIG. 5 can be performed by the extended checked baggage handling system (1) of FIG. 1b.

[0590] The above airport bus-linked checked baggage handling system (1) may be further configured to provide an extended checked baggage handling service that transports the user's checked baggage from the arrival airport to a desired location. The user's desired location may be a location different from the user's next destination.

[0591] In various embodiments of the present application, the airport bus-linked checked baggage handling system and method are configured to minimize transport to the wrong location by using an RFID tag (700) to multi-check baggage bags to be transported to a different location from the user.

[0592] Specifically, the airport bus-linked baggage handling method comprises: a step (S410) of obtaining information on a tourist destination to be visited by the user after the aircraft lands and information on a lodging place where the user will stay by means of the user's terminal device (100); and a step (S1100) of generating a list of user groups in the server (300) so that the baggage bags of the members of the list are transported to the lodging place and the members of the list visit the tourist destination, and transmitting the list to the staff terminal device (1210) of a specific tourist bus and the staff terminal device (1200) of a specific transport vehicle, respectively.

[0593] The tourist destination information indicates the tourist destinations the user will visit after the aircraft lands. The above tourist destination information may indicate the first place the user visits after the aircraft lands, or a series of places visited consecutively.

[0594] The above tourist location information may be a place in a country different from the country where the boarding airport is located, but is not limited thereto, and may be a place in the same country as the boarding airport.

[0595] The accommodation location information may indicate the accommodation location where the above user will stay.

[0596] These place information may be a location, address, place name, or other information indicating the place.

[0597] When the user's terminal device (100) confirms the mobile airport bus boarding pass, it can provide a UI screen for inputting tourist information and accommodation information to induce user input of tourist information and accommodation information (S410).

[0598] Then, the server (300) can associate the user’s account information (e.g., profile, user ID) associated with the user’s terminal device (100) with the user’s tourist location information and accommodation location information. Additionally, the server (300) can generate the user’s usage history including the user’s tourist location information and accommodation location information (S410).

[0599] If the usage history includes information on tourist destinations and accommodations, this user may be managed as a user who used the checked baggage handling service in the stage after aircraft landing, as described below.

[0600] FIG. 16 is a detailed flowchart of the process of generating a list of user groups according to various embodiments of the present application and transmitting it to the employee terminal device (1210) of a tour bus and the employee (1200) of a transport vehicle, respectively.

[0601] Referring to FIG. 16, the step (S1100) of generating a list of user groups and transmitting it to the staff terminal device (1210) of a specific tour bus and the staff terminal device (1200) of a specific transport vehicle, respectively, comprises: a step (S1110) of grouping users with the same tourist destination and accommodation location based on the user's tourist destination and accommodation location information for users who boarded the same aircraft; a step (S1120) of associating the generated user group with a specific tour bus on which the members of the user group board, and a step (S1120) of associating the generated user group with a specific transport vehicle that transports the luggage of the members of the user group; a step (S1130) of generating a list of user groups for each user group, including user identification information, the user's profile information, tourist destination information, accommodation location information, luggage tag information, and the UID of an RFID tag; and a step (S1140) of generating luggage transport details for each user based on the association relationship between the user group and the transport vehicle. The method may include the step (S1150) of transmitting the list of the generated user groups to the employee terminal device (1210) of the specific tour bus associated with the user groups and the employee terminal device (1200) of the specific transport vehicle.

[0602] To this end, terminal devices (not shown) of tour bus and transport vehicle operating companies providing tour buses and transport vehicles in the aforementioned destination area may pre-register the places or regions where the tour bus and transport vehicle can operate, and the boarding location (or departure location) as product vehicles on the server (300). Specifically, the terminal device of the tour bus operating company may register the tour bus's vehicle type, mileage, year of manufacture, number of passengers, places or regions where the tour bus can operate, and the boarding location (or departure location) as product vehicle information on the server (300). The terminal device of the transport vehicle operating company may also register the transport vehicle's vehicle type, mileage, year of manufacture, cargo capacity, places or regions where the tour bus can operate, and the boarding location as product vehicle information on the server (300).

[0603] The server (300) can search for and group users who have the same pair of tourist destinations and accommodations based on the user-specific tourist destination and accommodation information obtained and stored in step (S410) (S1110). The server (300) generates a list of user groups having various information associated with the user, such as user identification information, user profile information, tourist destination information, accommodation information, luggage tag information, and UID of the RFID tag as list items for each group.

[0604] The server (300) can select a specific tour bus to transport a user group from among the registered tour buses. The server (300) selects a specific tour bus from among a plurality of registered tour buses according to input from the server operator or a preset selection rule, and maps the selected specific tour bus to be in charge of the user group (S1130).

[0605] The server (300) can select a specific transport vehicle from among the registered transport vehicles to transport the luggage bag of the user group. The server (300) selects a specific transport vehicle from among a plurality of registered transport vehicles according to the input of the server operator or a preset selection rule, and maps the selected specific transport vehicle to be in charge of the luggage bag of the user group (S1130).

[0606] The server (300) can transmit a contract request to a terminal device of a tour bus operator providing a specific selected tour bus and a terminal device of a transport vehicle operator providing a specific transport vehicle, and confirm the selection by receiving a response of consent to the contract request. The contract request includes a driving schedule, a driving location (e.g., a tourist location or an accommodation location), etc.

[0607] The server (300) can update the usage history of member users belonging to the user group based on the relationship between the user group and the tour bus. Information indicating the tour bus scheduled to be boarded (e.g., identification information of the tour bus (e.g., vehicle registration number, etc.)) may be added to the usage history.

[0608] Additionally, the server (300) may generate a transport history for the user's luggage to separately manage the movement status of the luggage bag (S1140). When a user requests a set of luggage bags consisting of multiple luggage bags as checked baggage, multiple transport historys pointing to each of the multiple luggage bags may be generated (S1140). The transport history may include at least one of the following: a luggage linkage code corresponding to the luggage bag, an airline's luggage identification code, and a UID, as well as transport schedule information. The transport schedule information may include the aircraft arrival time and the luggage bag's transport destination (i.e., accommodation location).

[0609] Additionally, the airport bus-linked checked baggage handling method comprises the step (S1200) of transmitting a service schedule notification message to the user's terminal device (100), the message including information on the boarding location of a specific tour bus to be boarded by the user and information on the destination of a specific transport vehicle to be transported to the user's baggage. The service schedule notification message may be transmitted prior to the user's boarding of the tour bus.

[0610] The aforementioned service schedule notification message describes the identification information of the tour bus, the boarding location, and the destination of the tour bus. Additionally, the aforementioned service schedule notification message describes the destination of the specific transport vehicle where the user's luggage will be transported. - The destination information of the aforementioned specific transport vehicle indicates the location of the user's accommodation. The aforementioned service schedule notification message informs of the scheduled implementation status of the checked baggage handling service at the arrival area.

[0611] In addition, in the airport bus-linked checked baggage handling system (1) described above, the employee terminal device (1200) of the transport vehicle provides a list of user groups assigned to the employee in charge of the transport vehicle. Before loading a baggage bag onto the transport vehicle, the employee can verify whether it is a baggage bag that is actually to be loaded onto the transport vehicle by comparing the baggage tag information (e.g., baggage linkage code, baggage identification code, etc.) within the indicated user group with the baggage tag information on the paper label sticker attached to the baggage bag(s) to be loaded onto the transport vehicle.

[0612] In addition, the airport bus-linked baggage handling system (1) can be configured to additionally verify whether the baggage bags are baggage bags that are actually to be loaded onto a transport vehicle through an RFID tag (700).

[0613] Specifically, the airport bus-linked checked baggage handling method may include the step (S1300) of transporting and handling the baggage bags of a member of the user group based on the list of the user group.

[0614] FIG. 17 is a detailed flowchart of the process of transporting luggage bags of a user group in a transport vehicle according to various embodiments of the present application.

[0615] The employee terminal device (1200) of the above-mentioned specific transport vehicle can process the transport of luggage bags by interacting with an RFID tag (700) through an RFID reader module connected to the terminal device (1200).

[0616] Referring to FIG. 17, the step of transporting a luggage bag of a member of the user group (S1300) comprises: a step of obtaining luggage tag information associated with the member, a UID of an RFID tag, user identification information, and luggage destination information based on the received list of the user group (S1310) - the luggage destination information indicates the accommodation location of the user -; a step of reading tag information stored in an RFID tag attached to the luggage bag by means of an RFID reader module connected to an employee terminal device (1200) of the specific transport vehicle for each of at least one luggage bag to be loaded onto the specific transport vehicle (S1320); a step of first verifying by comparing the read tag information with the luggage tag information of the user obtained from the list of the user group (S1330); and a step of second verifying by comparing the UID of the RFID tag with the UID associated with the luggage bag stored in the server (300) (S1340). The method may include: a step (S1350) of generating a transportation processing result based on a comparison verification result for the user’s luggage bag and transmitting it to the server (300) - the comparison verification result includes a first verification result and a second verification result -; a step (S1360) of updating the user’s usage history based on the transportation processing result for the user’s luggage bag at the server (300); a step (S1370) of transmitting a first status notification message describing the transportation status of the user’s luggage bag to the user’s terminal device (100) if the comparison verification result is normal, and a step (S1380) of transmitting a second status notification message describing the transportation status of the user’s luggage bag and the luggage storage location to the user’s terminal device (100) corresponding to the abnormal comparison verification result.

[0617] The employee terminal device (1200) of a specific transport vehicle can obtain various information associated with the members based on the received list of the user group, such as baggage tag information, UID of the RFID tag, user identification information, and baggage destination information (S1310).

[0618] Similar to the above step (S750), the employee terminal device (1200) of a specific transport vehicle can also obtain tag information stored in the RFID tag (700) attached to the user's luggage bag (400) (S1320).

[0619] The employee terminal device (1200) of the above-mentioned specific transport vehicle compares the tag information read from the RFID tag (700) with the user's baggage tag information in the list. If these pieces of information match each other, the first verification result indicates normal, and conversely, if they do not match, the second verification result indicates abnormal. It is determined to be normal only when various items within the baggage tag information, such as the baggage linkage code and the airline's baggage identification code, all match each other (S1330).

[0620] The employee terminal device (1200) of the above-mentioned specific transport vehicle can directly obtain a UID from an RFID tag (700), request the server (300) to obtain a previously stored UID associated with a user in the list, and compare these two UIDs. If the two UIDs match as a result of the comparison, a secondary verification result indicating normality is generated, and if they do not match, a secondary verification result indicating abnormality is generated (S1340).

[0621] If at least one of the first and second check results indicates abnormality, the final comparison check result also indicates abnormality. Only when both the first and second check results indicate normal does the final comparison check result also indicate normality (S1350).

[0622] The employee terminal device (1200) of the above specific transport vehicle generates a transport processing result indicating transport success processing when the final comparison verification result is normal (S1350). When the final comparison verification result is abnormal, it generates a transport processing result indicating transport failure processing (S1350).

[0623] The transport processing results may indicate transport success or transport failure. Transport success includes the vehicle's departure location, destination, and transport processing time. Transport failure includes the vehicle's departure location, destination, storage location of the failed baggage, and transport processing time (i.e., time of failure recognition).

[0624] The transportation processing result based on the final comparison verification result is transmitted to the server (300) (S1350). The server (300) can update the transportation history for the luggage bag based on the transportation processing result (S1360).

[0625] The server (300) can report the transport status of the luggage bag to the user in real time by sending a first status notification message or a second status notification message (S1370, S1380).

[0626] Through this process (S1300), multiple checks are possible using label stickers and RFID tags (700), thereby further increasing the reliability of the luggage bag transport via transport vehicles.

[0627]

[0628] Additionally, the airport bus-linked checked baggage handling method includes the step (S1400) of processing the boarding of members of the user group based on the list of the user group by the employee terminal device (1210) of the specific tour bus.

[0629] FIG. 18 is a detailed flowchart of the process of handling members of a user group boarding a tour bus according to various embodiments of the present application.

[0630] Referring to FIG. 18, the step of processing the boarding of members of the user group (S1400) may include: a step of performing a first boarding confirmation by comparing a list of the user group and a list of identity information of users who boarded the specific tour bus by the employee terminal device (1210) of the specific tour bus (S1410); a step of transmitting the result of the boarding confirmation of the user group for the specific tour bus from the employee terminal device of the specific tour bus to the server (300) (S1430) - the boarding confirmation result includes the user's identification information, the identification information of the specific tour bus, and the time of the user's boarding confirmation -; a step of updating the user's usage history by the server (300) with the boarding confirmation result (S1450); and a step of transmitting a non-boarding inquiry message to the terminal device (100) of the non-boarding user by the server (300) based on the boarding confirmation result (S1470).

[0631] The employee terminal device (1210) of a specific tour bus can display a list of user groups scheduled to board the specific tour bus and receive user input from an employee indicating whether the members of the displayed list are boarding or not, and process the boarding status. When user input from an employee indicating boarding for a specific member is received, the specific member is processed as having completed boarding, and when user input from an employee indicating non-boarding for a specific member is received, the specific member is processed as having not completed boarding (S1410). The employee terminal device (1210) of the specific tour bus generates a boarding processing list indicating boarding completion / non-completion for each user within the group as a boarding confirmation result and transmits it to the server (300) (S1430). Then, the server (300) updates the user's usage history with the boarding confirmation result to reflect the fact of boarding completion / non-completion in the user's usage history (S1450).

[0632] The non-boarding inquiry message of the above step (S1470) may include user profile information, baggage tag information, and the operation schedule of the tour bus. Through the non-boarding inquiry message, the system (1) can finally confirm whether the user is not on board.

[0633] This non-boarding inquiry message is similar to the non-boarding inquiry message sent to a user who did not board the airport bus in step (S300). As the transmission process has been described in detail above, a detailed explanation is omitted.

[0634] In some embodiments, the step (S1400) may further include a step (S1430) of performing a second boarding confirmation for each boarding user based on GPS data received from the user's terminal device and GPS data from the tour bus staff terminal device. In this case, the boarding confirmation result of the step (S1430) includes a first boarding confirmation result and a second boarding confirmation result. Since the process of obtaining the second boarding confirmation result is similar to the step (S500), a detailed description is omitted.

[0635] For example, let's assume the arrival airport is Yangyang Airport, the tourist destinations are a series of places consisting of Yangyang Traditional Market, Naksansa Temple, Mulchi Port, the entrance to Seoraksan National Park, and Hanwha Resort Seorak Waterpia in that order, and the accommodation is Hanwha Resort Seorak Waterpia.

[0636] The luggage bags of the user group are loaded onto a specific transport vehicle. At the time of loading, the employee terminal device (1200) of the specific transport vehicle compares the information on the label sticker and luggage tag (700) attached to the luggage bag with the information within the user group's list, and luggage bags confirmed to be free of defects are loaded and processed as successful transport (S1300). Then, the luggage bags are transported directly from Yangyang Airport to Hanwha Resort Seorak Waterpia.

[0637] Meanwhile, when the user group arrives at Yangyang Airport, they can immediately board a specific tour bus that operates to a series of locations and check the transport status of their luggage in real time while enjoying the tour (S1400). In other words, the user group can start the tour experience immediately upon arrival at Yangyang Airport and does not need to go through the hassle of stopping by their accommodation first to leave their luggage. As a result, the user's tour route and wasted time after arrival from the aircraft are minimized.

[0638] The above airport bus-linked baggage handling system and method can manage baggage received by the system more reliably through RFID tags (700).

[0639]

[0640] In alternative embodiments, the airport bus-linked checked baggage handling system (1) and method may also be implemented in a form without RFID. Specifically, such a system (1) acquires data for an airport bus to be used by a user to arrive at an airport to board an aircraft, generates a baggage bag image of the user’s baggage bag to be loaded onto the aircraft and an image of the inside of the bag showing the inside of the user’s baggage bag, analyzes the images to calculate the user’s baggage data, and transmits it to a server - the baggage data includes baggage data and baggage item data -; An airport bus terminal device that verifies the identity of the user by analyzing a facial image of the user boarding the airport bus, verifies the airport bus boarding pass of the user boarding the airport bus, authenticates the user's intention to board an aircraft based on the user's GPS data received from the verified user's terminal device and the airport bus's GPS data, requests and obtains the user's mobile aircraft boarding pass data from the user's terminal device or server after the user's intention to board the aircraft has been authenticated, and generates a baggage linkage code corresponding to the user's mobile aircraft boarding pass based on the user's baggage data - the user's mobile aircraft boarding pass data is stored in advance on a server or on the user's terminal device in association with the user's account information through a pre-check-in process, and the baggage linkage code is a code different from the airline baggage identification code -;The system may include a server configured to generate and manage baggage tag information, including a baggage linkage code and an airline's baggage identification code, in a digital form based on a baggage linkage code transmitted from a terminal device of the airport bus, and to receive a security inspection result including a second baggage bag image captured by the airport's baggage inspection system and a reading result data obtained from inspection source data for the baggage bag, wherein the reading result data includes an identification result for at least one item contained in the user's baggage bag, and to associate the reading result data for the baggage bag with the user's baggage linkage code based on the matching result between the second baggage bag image of the baggage inspection system and the user's baggage bag image, and the matching result between the reading result data of the baggage inspection system and the user's baggage item data, and to transmit a baggage check-in notification message to the user's terminal device based on an inspection result message for the user's baggage received from the baggage inspection system.

[0641] In this case, the step (S730) of receiving the user’s baggage tag information transmitted to the RFID tag from the RFID reader module and the UID of the RFID tag, associating them with each other, and updating the user’s usage history having the baggage tag information associated with the UID of the RFID tag, and other related steps may be omitted.

[0642]

[0643] When implementing an embodiment of the present invention using hardware, ASICs (application specific integrated circuits) or DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), etc. configured to perform the present invention may be provided in the processor of the present invention.

[0644] Meanwhile, the method described above can be written as a program executable on a computer and can be implemented on a general-purpose digital computer that operates said program using a computer-readable medium. Additionally, the structure of the data used in the method described above can be recorded on a computer-readable storage medium through various means. Program storage devices that may be used to describe a storage device containing executable computer code for performing various methods of the present invention should not be understood to include transient objects such as carrier waves or signals. The computer-readable storage medium includes storage media such as magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.).

[0645] The embodiments described above are combinations of the components and features of the present invention in a specific form. Each component or feature should be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, it is possible to construct embodiments of the present invention by combining some components and / or features. The order of operations described in the embodiments of the invention may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment. It is obvious that embodiments may be constructed by combining claims that do not have an explicit citation relationship in the claims, or that they may be included as new claims through amendments made after filing.

[0646] It will be apparent to those skilled in the art that the present invention may be embodied in other forms without departing from the technical spirit and essential features of the present invention. Accordingly, the above embodiments should be considered in all illustrative aspects rather than as a limiting one. The scope of the present invention shall be determined by a reasonable interpretation of the appended claims and all possible variations within the equivalent scope of the present invention.

[0647] The embodiments of the present application have industrial applicability in the field of airport bus technology in that they resolve the inconvenience of the conventional checked baggage handling process, in which a user must take their baggage out of the airport bus and drag it to the airline's check-in desk.

Claims

1. In an airport bus-linked checked baggage handling system that enhances service reliability by applying RFID technology to checked baggage received via AI-based recognition and GPS coordinates, A user terminal device that acquires boarding pass data for an airport bus to be used by a user to arrive at an airport to board an aircraft, generates a luggage bag image of the user’s luggage bag to be loaded onto the aircraft and an image of the interior of the user’s luggage bag, analyzes the images to calculate the user’s luggage data, and transmits it to a server; wherein the luggage data includes luggage bag data and luggage item data; An airport bus terminal device that verifies the identity of the user by analyzing a facial image of the user's face taken while boarding the airport bus, verifies the airport bus boarding pass of the user who has boarded the airport bus, authenticates the user's intention to board an aircraft based on the user's GPS data received from the terminal device of the verified user and the airport bus's GPS data, requests and obtains the mobile aircraft boarding pass data of the user whose intention to board the aircraft has been authenticated from the user's terminal device or server, and generates a baggage linkage code corresponding to the user's mobile aircraft boarding pass based on the user's baggage data - the user's mobile aircraft boarding pass data is stored in advance on a server or on the user's terminal device in association with the user's account information through a pre-check-in process, and the baggage linkage code is a code different from the airline baggage identification code; and A server configured to generate and manage baggage tag information including a baggage linkage code and an airline's baggage identification code in a digital form based on a baggage linkage code transmitted from a terminal device of the airport bus, receive a security inspection result including a second baggage bag image captured by the airport's baggage inspection system and a reading result data obtained from inspection source data for the baggage bag, wherein the reading result data includes an identification result for at least one item contained in the user's baggage bag, associate the reading result data for the baggage bag with the user's baggage linkage code based on the matching result between the second baggage bag image of the baggage inspection system and the user's baggage bag image, and the matching result between the reading result data of the baggage inspection system and the user's baggage item data, and transmit a baggage check-in notification message to the user's terminal device based on an inspection result message for the user's baggage received from the baggage inspection system; and A staff terminal device connected to an RFID reader module to store the above baggage tag information in an initialized RFID tag Includes, The server is further configured to receive the user's baggage tag information transmitted to the RFID tag from the RFID reader module and the UID of the RFID tag, associate them with each other, and update the usage history of the user having the baggage tag information associated with the UID of the RFID tag. Checked baggage handling system.

2. In paragraph 1, the RFID tag is, Characterized by being configured to encrypt the user's baggage tag information received through the above-mentioned RFID reader module and to store the encrypted user's baggage tag information in memory. Checked baggage handling system.

3. In Paragraph 1, The employee terminal device of the specific transport vehicle is configured to read tag information stored in an RFID tag attached to a luggage bag through the RFID reader module, perform a first verification by comparing the read tag information with the luggage tag information of the user obtained from the user group list, and perform a second verification by comparing the UID of the RFID tag with the UID stored in the server. The above server is configured to verify the validity of the TOTP generated from the RFID tag upon a request from the employee terminal device, wherein the employee terminal device transmits a wireless signal requesting the generation of TOTP to the RFID tag (700) through the RFID reader module, and The processor of the above RFID tag generates a TOTP and transmits it as a response signal, and Receive the TOTP of the response signal through the RFID reader module, and The above employee terminal device transmits the delivered TOTP to the server, and the server verifies the validity of the TOTP, If TOTP is valid on the above server, update the user's baggage tag information, and Characterized by transmitting an RFID status notification message to the user's terminal device associated with the RFID tag when the validity verification result of the above TOTP is abnormal. Checked baggage handling system.

4. In Paragraph 1, The terminal device of the above airport bus is, Determining whether the airport bus has entered a pre-set designated section based on GPS data of the airport bus, wherein the designated section is a certain section including an airport stop within the entire operating section of the airport bus; When the above airport bus enters a pre-set designated section, determine whether the user's GPS location and the airport bus's GPS location match at at least one point on the designated section; Characterized by being configured to acknowledge the user's intention to board an aircraft when it is determined that the user's GPS location and the airport bus's GPS location match each other at at least one point on the designated section. Checked baggage handling system.

5. In Paragraph 1, The terminal device of the above user, in order to calculate the user's baggage item data, The above luggage bag image is applied to a first deep learning-based recognition model to recognize the user's luggage bag; Based on the recognition result of the luggage bag, the weight data of the user's luggage bag is retrieved via the Internet or a server; Applying the interior image of the user's bag to a deep learning-based segmentation model to segment the item areas where luggage appears in the interior image of the bag according to the luggage class, and extracting the segmented item areas as sub-images; The sub-image of the extracted item area is applied to a second deep learning-based recognition model to recognize the item appearing in the image inside the bag; Based on the result of recognizing the items, search for weight data of the items in the user's luggage bag via the Internet or a server; Characterized by being configured to calculate the user's luggage weight information based on the weight search results of the recognized luggage bag and the weight search results of the recognized items. Checked baggage handling system.

6. In Paragraph 5, The above user's terminal device is, If the class of the above-recognized item corresponds to a pouch, input of information regarding the items inside the pouch contained in the pouch is required, and Characterized by being further configured to retrieve weight data for said pouch based on information about the internal contents of the pouch entered and size data of the pouch calculated from the division result of the item area where the pouch appears. Checked baggage handling system.

7. In Paragraph 5, The above deep learning-based segmentation model is trained using a first training data set, and The first training data set above consists of a plurality of training samples, each training sample consists of training data and label data, and The above training data includes at least one training image, a sample image of an incomplete shape representing a training sample item having an incomplete shape, and a sample image of a complete shape representing an article having a complete shape as an article appearing in the sample image of the incomplete shape. The above label data is characterized by representing the actual value of the item type of the luggage appearing in the corresponding training image. Checked baggage handling system.

8. In Paragraph 5, The second deep learning-based recognition model comprises an input layer that inputs the sub-image as an input image, an encoder including one or more encoding blocks, a decoder including one or more decoding blocks, and a classification layer that recognizes and outputs an item appearing in the sub-image based on an encoding vector output from the decoder. The above second deep learning-based recognition model is, Divide the input sub-image into multiple patches - each patch has a fixed size, Convert each patch into a patch embedding vector, and Combine the position embedding value representing the location of each patch with the patch embedding vector of the corresponding patch, and The combined vector is input into the encoder to process and produce an encoding vector, and Recognize the item appearing in the sub-image based on the above encoding vector, and It is configured to output recognition results, and The encoder outputs class tokens as encoding vectors through a multi-head attention mechanism and is characterized by learning the relationships between multiple patches through the multi-head attention mechanism. Checked baggage handling system.

9. In Paragraph 8, The above-described second deep learning-based recognition model is characterized by a foundation model pre-trained with a specific dataset being fine-tuned to recognize incomplete or complete shaped items contained within a luggage bag. Checked baggage handling system.

10. In Paragraph 5, The above user's terminal device is, Based on the size information and division result of the above luggage bag, the size information of the item appearing in the item area is calculated, and It is configured to apply the corresponding sub-image showing the luggage and the size information of the item to the second deep learning-based recognition model, and The above second deep learning-based recognition model is, Converts normalized item size information into a size information tensor, generates luggage tensor data based on feature maps extracted from sub-images and the size information tensor, inputs the generated luggage tensor data into a fully connected layer as an intermediate operation result, and Characterized by being configured to recognize an item appearing in a sub-image having an incomplete shape by considering the characteristics of the luggage having an incomplete shape through the above-mentioned complete connection layer, Checked baggage handling system.

11. In Paragraph 1, The above server is, When a rejection message indicating that restricted checked baggage was detected during baggage inspection is received from the baggage inspection system, a first notification message including the baggage verification location and the baggage linkage code is transmitted to the user's terminal device based on the baggage linkage code within the rejection message. Characterized by being configured to transmit a second notification message, including a baggage management number issued by the baggage inspection system, to the user's terminal device when receiving a pass message from the baggage inspection system indicating that no restricted checked baggage was found during the baggage inspection. Checked baggage handling system.

12. In Paragraph 1, When the above user's terminal device calculates the baggage data, Check if the user's baggage data meets the pre-set airline check-in weight limit; Based on the user's baggage data, verify whether the user's baggage satisfies the pre-set checked baggage restriction guide; If the user's baggage falls within the checked baggage weight limit, a first warning message including checked baggage weight limit information is provided to the user; If items restricted from checked baggage are included among the recognized baggage items, a second warning message including the detection result of the restricted items is provided to the user; If the consignment weight limit is not met and no restricted items are identified, provide a pass message to the user; Characterized by being further configured to update the user's baggage data according to user input indicating the removal of baggage items entered after providing the first warning message or the second warning message above. Checked baggage handling system.

13. In Paragraph 1, If the terminal device of the aforementioned airport bus fails to verify identity as a result of analyzing the user's face image, The terminal device of the above airport bus outputs the fact of identity verification failure through the output device, and The above-mentioned user terminal device is characterized by being configured to capture a second face image through a UI screen capable of capturing the user's real-time face at the time of boarding, input the second face image into a pre-trained identity verification model to obtain the user's identity information, and transmit the obtained user's identity information to the server or the terminal device of the airport bus. Checked baggage handling system.

14. In paragraph 1, the server is, Characterized by being configured to transmit a third notification message, including the user's baggage linkage code and baggage verification location, to the user's terminal device when the baggage bag shown in the second baggage bag image of the baggage inspection system and the baggage bag shown in the user's baggage bag image do not match each other. Checked baggage handling system.

15. In Paragraph 1, The above server is, When the baggage linkage code of the above user is obtained, the baggage linkage code is associated with the user's account information and aircraft boarding pass information, and Characterized by being configured to transmit a linked service reception message including the user's baggage tag information to the user's terminal device. Checked baggage handling system.

16. A computer-readable recording medium having a program for performing a method of operation of a checked baggage handling system according to any one of paragraphs 1 through 15.

Images