A method, system and storage medium for acquiring passenger flow based on face recognition

Abstract

The present invention discloses a passenger flow acquisition method, system and storage medium based on face recognition, the method comprising: acquiring a video stream of passengers getting on and off a bus captured by a camera, each frame of the video stream being marked with a timestamp when the image was captured; marking a face area in each frame of the image, and converting the face in the marked area into a frontal face; classifying passengers as getting on and off a bus according to the moving track of the frontal face of the passenger getting on and off the bus in the video stream; matching the facial features of the passengers getting on and off the bus with the facial features of the passengers getting off the bus, and when the match is successful and the two facial features that are successfully matched correspond to different getting on and off stations, it is recorded as one passenger traffic volume, until all facial features are matched. The present invention improves the statistical accuracy of passenger traffic volume by performing image recognition after frontal face conversion.

Description

Technical Field

[0001] The present invention relates to the technical field of passenger traffic volume statistics, and in particular to a passenger flow acquisition method, system and storage medium based on face recognition used in public transportation commuter vehicles. Background Art

[0002] The existing statistics of passenger traffic volume (OD) on buses are generally calculated in the following ways:

[0003] 1. Use IC card swipe to count, but since most IC cards are only swiped once, it is good for counting the boarding stations and the number of people, but it is not possible to count the getting off the bus.

[0004] 2. Use ordinary binocular or structured light to count, but only count the number of people getting on and off the bus, but cannot count the specific time and station of each passenger getting on and off the bus. This makes it impossible to perform subsequent data statistical analysis, such as a passenger's daily commuting time period, commuting route, commuting peak of a certain line, etc.

[0005] 3. Use ordinary image recognition to perform passenger flow OD statistics. However, since there are many side images in the passenger images, the matching and recognition process is prone to errors, and the accuracy of the statistical data is not high.

[0006] That is, the existing passenger traffic volume (OD) statistics have low accuracy and cannot accurately obtain the boarding and alighting stations and times of each passenger, making it impossible to carry out subsequent big data statistical analysis.

[0007] Therefore, the prior art needs to be improved. Summary of the invention

[0008] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a passenger flow acquisition method, system and storage medium based on face recognition, aiming to improve the statistical accuracy of passenger traffic volume (OD) through frontal face conversion and image recognition technology.

[0009] To achieve the above object, the present invention adopts the following technical solutions:

[0010] In a first aspect, the present invention proposes a method for acquiring passenger flow based on face recognition, wherein the method comprises the following steps:

[0011] S10, obtaining a video stream of passengers getting on and off the bus captured by a camera, wherein each frame of the video stream is marked with a timestamp when the image was captured;

[0012] S20, marking a face region in each frame of the image, and converting the face in the marked region into a frontal face;

[0013] S30, tracking and identifying each converted frontal face in the video stream, and classifying the passengers as boarding passengers and alighting passengers according to the movement trajectory of the tracked frontal face of the same person in the video stream;

[0014] S40, assigning a UUID to each boarding passenger, extracting facial features from the frontal facial image of each boarding passenger, and obtaining the corresponding boarding station according to the timestamp of the boarding movement trajectory video stream of each boarding passenger, and storing the UUID, facial features, and boarding station corresponding to each boarding passenger in the first linked list;

[0015] S50, extracting facial features from the front face image of each passenger getting off the bus, and obtaining the corresponding getting off station according to the timestamp of the video stream of the getting off movement trajectory of each passenger getting off the bus, and storing the facial features and the getting off station corresponding to each passenger getting off the bus in a second linked list;

[0016] S60, matching the facial features in the first linked list with the facial features in the second linked list, when the match is successful and the two facial features correspond to different boarding and alighting stops, it is recorded as one passenger traffic volume, until all facial features are matched and the total passenger traffic volume is obtained.

[0017] In a second aspect, the present invention proposes a system, wherein the system also includes a memory, a processor, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the aforementioned method is implemented.

[0018] In a third aspect, the present invention proposes a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program implements the aforementioned method when executed.

[0019] It should be understood that within the scope of the present invention, the above-mentioned technical features of the present invention and the technical features specifically described below (such as implementation methods) can be combined with each other to form new or preferred technical solutions. Due to space limitations, they will not be described one by one here.

[0020] The passenger flow acquisition method based on face recognition of the present invention converts the face in the video stream marked with timestamp into a frontal face for tracking and recognition, and classifies the passengers into boarding passengers and disembarking passengers according to the boarding and disembarking movement trajectory of the frontal face in the video stream, then extracts the facial features of the classified boarding passengers and disembarking passengers and obtains the boarding and disembarking stations, and then matches the facial features of the boarding passengers and disembarking passengers respectively. When the match is successful and the boarding and disembarking stations corresponding to the two successfully matched facial features are different, it is recorded as one passenger traffic volume, until all facial features are matched and the total passenger traffic volume is obtained. Since the present invention converts all the acquired video stream images into frontal faces before tracking, identifying and matching, the recognition accuracy is greatly improved. At the same time, the boarding and disembarking stations and boarding and disembarking times of each passenger can be accurately obtained through the timestamps in the moving trajectory video stream, providing accurate data support for subsequent statistical analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0022] Figure 1 This is a flow chart of a first embodiment of a method for acquiring passenger flow based on face recognition according to the present invention;

[0023] Figure 2 It is a schematic diagram of the process of multi-view face frontalization of the present invention;

[0024] Figure 3 Schematic diagram of the conversion process of multi-view frontalization of the present invention;

[0025] Figure 4 This is a schematic diagram of the deep learning frontalization of a single side face according to the present invention;

[0026] Figure 5 It is a schematic diagram of the process of classifying boarding passengers and disembarking passengers according to the present invention;

[0027] Figure 6 A schematic diagram of setting a boarding determination line and a disembarking determination line in a vehicle according to the present invention;

[0028] Figure 7 A schematic diagram of the movement trajectory of the passenger's face region in the vehicle according to the present invention;

[0029] Figure 8 The present invention is a schematic diagram of the process of obtaining the boarding and alighting points of passengers. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] like Figure 1 As shown, the passenger flow acquisition method based on face recognition of the present invention specifically comprises the following steps:

[0032] S10, obtaining a video stream of passengers getting on and off the bus captured by a camera, wherein each frame of the video stream is marked with a timestamp when the image is captured.

[0033] The video stream of a passenger is a collection of multiple consecutive frames of images, and the timestamp of each frame is the acquisition time of each image, which can be a camera mark or a system mark. The present invention uses a binocular camera to simultaneously shoot the same passenger from multiple angles or uses a single camera to shoot.

[0034] S20, marking a face area in each frame of the image, and converting the face in the marked area into a frontal face.

[0035] The video stream image of the passenger captured by the camera includes the face, other parts of the body and the background. To facilitate subsequent face recognition, the face area is first marked in the entire image, such as marking a rectangular area containing the face in the entire image.

[0036] At the same time, since passengers getting on and off the bus do not necessarily face the camera directly, the images in the video stream may all be side faces or most of them are side faces. If the side images are directly used for face recognition as in the prior art, it is easy to cause misalignment in the subsequent face tracking and recognition, such as recognizing the face of passenger A as the face of passenger B. This will cause errors in the subsequent boarding and alighting stations, boarding and alighting times, and the entire passenger traffic volume (OD), with low error rate and accuracy. Therefore, the present invention converts the face in the video stream into a front face, and then uses the front face for tracking and recognition to improve the recognition accuracy, so as to ensure the accuracy of the subsequent traffic volume (OD) statistics.

[0037] Specifically, Figure 2 As shown, as an implementation mode, in step S20 of the embodiment of the present invention, the face in the marked area is converted into a frontal face through a multi-view frontalization method, which specifically includes the following steps:

[0038] S2011, obtaining facial images of the same passenger taken at different angles by two cameras at the same time.

[0039] like Figure 3 As shown, two cameras are used, camera 1 and camera 2, to respectively shoot facial images of the same passenger at right and left angles at the same time. After shooting, facial images tilted to the left and right and facial images tilted to the right are obtained respectively.

[0040] S2012, rotating the face leaning to the right in the face image to the left and the face leaning to the left to the right through affine transformation, to obtain two frontal face images respectively.

[0041] Affine transformation is to restore the entire side face image to a front face image by stretching one side of the face and compressing the other side of the face during the rotation of a single face image. Using the front face image for tracking and recognition greatly improves the accuracy of tracking and recognition compared to the existing technology of directly tracking and recognizing the side face image.

[0042] S2013, averaging the frontal faces in the two frontal face images to obtain a corrected frontal face.

[0043] Since the affine transformation converts and restores a single side face image, the restored result may have errors such as fatter or thinner faces. The embodiment of the present invention averages the two front faces restored by the affine transformation to eliminate the errors after stretching and compression, thereby obtaining a more accurate and close to the passenger's real front face.

[0044] In step S2012 of the embodiment of the present invention, the frontal face conversion is performed by using both the stretching formula and the compression formula when the face image is rotated left or right through affine transformation.

[0045] Because the side face of a person has a rotation angle relative to the front face, one side of the face is stretched relative to the front face and the other side is compressed. Therefore, when the side face is restored to the front face, the compressed facial area needs to be stretched at the same time.

[0046] Assume that the pixel size of the original image before affine transformation is m×n, and (x, y) represents a pixel point on the image.

[0047] The stretching formula is:

[0048]

[0049] The compression formula is:

[0050]

[0051] Among them, θ represents the angle of facial rotation, α represents half of the angle between the two cheeks of the face, α∈[70°,85°], θ<70°, nose represents the horizontal pixel coordinate of the position of the nose tip in the original image, m represents the number of pixels in the width direction of the original image, n represents the number of pixels in the height direction of the original image, δ is the relaxation factor, δ takes a value between 0 and 1, and δ is used to adjust the effect of the stretching and compression algorithm.

[0052] As another embodiment, Figure 4 As shown, in step S20 of the present invention, the face in the marked area is converted into a frontal face, and the method of converting the face into a frontal face by deep learning of a single side face can be used, which specifically includes the following steps:

[0053] S2021, collecting video stream images of passengers getting on and off the vehicle at the door position, wherein the video stream image of each passenger includes multiple facial gestures.

[0054] S2022, extract all facial images from the video stream, group the facial images according to different passengers, and mark the facial area as well as the front face and the side face in the grouped images as a training data set.

[0055] S2023, adopting a conditional generative adversarial network model and using the training data set for training.

[0056] The conditional generative adversarial network model, i.e., the (cGAN) model, can be connected and other models can be used to train the data set.

[0057] S2024, using the trained model, performs frontal face prediction on the facial images of people getting on and off the vehicle collected during actual vehicle operation, and outputs the frontal face images.

[0058] After the face image in the video stream is transformed into a frontal face in the above manner, the frontal face is used for recognition, specifically:

[0059] S30, tracking and identifying each converted frontal face in the video stream, and classifying the passengers as boarding passengers and alighting passengers according to the movement trajectory of the tracked frontal face of the same person in the video stream.

[0060] That is, before the present invention counts the passenger traffic volume, it is necessary to classify all passengers in the video stream and separate the boarding passengers and the disembarking passengers.

[0061] like Figure 5 As shown, step S30 of the embodiment of the present invention specifically includes the following steps:

[0062] S301, identifying and comparing the converted frontal face in the face region of the current frame image with the converted frontal face in the face region of the previous frame image.

[0063] Frontal face recognition generally includes steps such as face detection, face alignment, and face feature representation. Frontal face recognition can use the SeetaFace open source library.

[0064] S302, if the compared frontal faces are of the same person, the position of the face area in the current frame image is associated with the position of the face area in the previous frame image and recorded; if the compared frontal faces are not of the same person, the new frontal face is tracked and identified again directly starting from the current frame.

[0065] The associated record is to record the position of the face area in multiple consecutive frames of the same person in the same set of data. For example, in the position data row of passenger A, the position of the face area in the first frame, the position of the face area in the second frame, and so on, the position of the face area in the Nth frame are recorded in sequence. If the front faces of the two frames are not the same person, the passenger corresponding to the face in the current frame is recorded as passenger B, and then the front face of passenger B is tracked and recognized again, and the position data of its face area is recorded in sequence.

[0066] The position data of the face region is the coordinate data of the center point of the face region.

[0067] S303, repeating steps S301 to S302 to obtain the movement trajectory of the face area of ​​the same person.

[0068] After obtaining the position data of the face area in multiple frames of images of the same person, the moving trajectory of the passenger's face area in the car is obtained, and the various positions can be connected to form a trajectory line.

[0069] S304: Mark a get-off determination line at a position close to the vehicle door in the image of the video stream, and mark a get-on determination line inside the get-off determination line.

[0070] like Figure 6 As shown, the boarding and disembarking determination line refers to two positions of the door area captured by the camera. For example, the present invention uses the point where the boarding step is just taken as the boarding determination line, and the inner side of the boarding determination line, the point where the boarding passes the card reader position, is used as the boarding determination line.

[0071] The doors are divided into front doors and middle doors. In the embodiment of the present invention, the front door, rear door and middle door are respectively set with boarding determination line and alighting determination line. Because in most cases, the front door is used for boarding and the middle door is used for alighting, but some passengers board and alight nearby, so the trajectory determination needs to be performed on both doors in the embodiment of the present invention.

[0072] S305: Determine the passenger as a boarding passenger or an alighting passenger based on the position change of the movement trajectory of the same person relative to the boarding determination line and the alighting determination line.

[0073] The step S305 of the embodiment of the present invention specifically includes:

[0074] If the passenger's movement trajectory first crosses the getting-off determination line and then crosses the getting-on determination line, he / she is classified as a getting-on passenger; if the passenger's movement trajectory first crosses the getting-off determination line and then crosses the getting-off determination line, he / she is classified as a getting-off passenger. Figure 7 As shown in , the passenger's facial area first crosses the getting-off determination line and then crosses the getting-on determination line, so the passenger corresponding to the face is determined to be a getting-on passenger.

[0075] After the passengers are classified into different categories, the linked lists of passengers getting on and off the bus are created separately, as follows:

[0076] S40, assign a UUID to each passenger who gets on the bus, extract the facial features from the frontal facial image of each passenger who gets on the bus, and obtain the corresponding boarding station based on the timestamp of the video stream of the boarding movement trajectory of each passenger who gets on the bus, and store the UUID, facial features, and boarding station corresponding to each passenger who gets on the bus into the first linked list.

[0077] The passenger's UUID is the passenger's unique identification code. Each passenger corresponds to a unique UUID to prevent errors in the matching process.

[0078] S50, extracting facial features from the frontal facial image of each passenger getting off the bus, and obtaining the corresponding getting off station according to the timestamp of the video stream of the getting off movement trajectory of each passenger getting off the bus, and storing the facial features and getting off station corresponding to each passenger getting off the bus in a second linked list.

[0079] like Figure 8 As shown, obtaining the boarding and alighting sites in steps S40 and S50 specifically includes:

[0080] S71, obtaining the timestamp of the video stream of the passenger's boarding trajectory or obtaining the timestamp of the video stream of the passenger's alighting trajectory.

[0081] S72, obtaining the station announcement information from the bus station announcement device or from the bus management network platform, and matching the time in the station announcement information with the timestamp to obtain the passenger's boarding station or alighting station.

[0082] The station announcement information of the present invention can be obtained from the vehicle station announcement device, and the time of each station announcement can be obtained, and the passenger's boarding location can be matched according to the time. Of course, because the vehicle can only transmit the passenger's boarding and alighting time to the bus management network platform, the present invention can also be that the bus management network platform itself matches the boarding and alighting stations according to the station announcement information received by the station announcement device.

[0083] After the first linked list of boarding passengers and the second linked list of disembarking passengers are established, subsequent face matching can be performed, specifically:

[0084] S60, matching the facial features in the first linked list with the facial features in the second linked list, when the match is successful and the two facial features correspond to different boarding and alighting stops, it is recorded as one passenger traffic volume, until all facial features are matched and the total passenger traffic volume is obtained.

[0085] When the faces of the boarding and disembarking passengers are matched successfully, the boarding and disembarking stations are matched by the boarding and disembarking time. Only when the boarding and disembarking stations are different can it be counted as a valid passenger traffic volume record. This can filter out some passengers who get on the wrong bus and get off immediately. At the same time, the driver's boarding and disembarking can also be filtered out.

[0086] In the step S60 of the embodiment of the present invention, the facial features in the first linked list and the facial features in the second linked list are matched by calculating the similarity between the two facial features to be matched. When the calculated similarity is greater than a preset threshold, it indicates that the match is successful. For example, if the preset threshold is T, when the calculated similarity is greater than T, it indicates that the match is successful.

[0087] The positive facial features of the passenger flow acquisition method based on facial recognition of the present invention can be stored in the background database, which can not only record the boarding and alighting station records of each passenger's single trip, but also retrieve and match the boarding and alighting station records of each passenger for one day, as well as the boarding and alighting station records of each day in the database, identify commuting passengers, and extract commuting time periods, commuting routes and stations, so as to provide accurate data support for subsequent statistical analysis.

[0088] The passenger flow acquisition method based on face recognition in the embodiment of the present invention converts the face in the video stream marked with a timestamp into a frontal face for tracking and recognition, and classifies the passengers into boarding passengers and disembarking passengers according to the boarding and disembarking movement trajectory of the frontal face in the video stream, then extracts the facial features of the classified boarding passengers and disembarking passengers and obtains the boarding and disembarking stations, and then matches the facial features of the boarding passengers and disembarking passengers respectively. When the match is successful and the boarding and disembarking stations corresponding to the two successfully matched facial features are different, it is recorded as one passenger traffic volume, until all facial features are matched and the total passenger traffic volume is obtained. Since the present invention converts all the acquired video stream images into frontal faces before tracking, identifying and matching, the recognition accuracy is greatly improved. At the same time, the boarding and disembarking stations and boarding and disembarking times of each passenger can be accurately obtained through the timestamps in the moving trajectory video stream, which is convenient for subsequent commuting data statistical analysis.

[0089] The present invention further proposes a system, which further includes a memory, a processor, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the above method is implemented.

[0090] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the asynchronous message processing terminal device.

[0091] The main control module may include but is not limited to a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples based on the system and do not constitute a limitation on the main control module. The main control module may include more or fewer components than the above components, or a combination of certain components, or different components. For example, the main control module may also include input and output devices, network access devices, buses, etc.

[0092] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the device, and various interfaces and lines are used to connect the various parts of the entire main control module.

[0093] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to use (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0094] The present invention further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program implements the above method when executed.

[0095] If the module / unit integrated in the passenger flow acquisition method based on face recognition of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The specific implementation of the computer-readable storage medium of the present invention is basically the same as the above-mentioned embodiments of the passenger flow acquisition method based on face recognition, and will not be repeated here.

[0096] It should be noted that the embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art may understand and implement it without paying any creative work.

[0097] The above description is only an example to clearly illustrate the present invention, and does not limit the patent scope of the present invention. It is impossible to list all the implementation methods here. All equivalent structural changes made by using the contents of the technical solution of the present invention under the concept of the present invention, or directly / indirectly used in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for acquiring passenger flow based on face recognition, characterized in that: The method comprises the following steps: S10, obtaining a video stream of passengers getting on and off the bus captured by a camera, wherein each frame of the video stream is marked with a timestamp when the image was captured; S20, marking a face region in each frame of the image, and converting the face in the marked region into a frontal face; S30, tracking and identifying each converted frontal face in the video stream, and classifying the passengers as boarding passengers and alighting passengers according to the movement trajectory of the tracked frontal face of the same person in the video stream; S40, assigning a UUID to each boarding passenger, extracting facial features from the frontal facial image of each boarding passenger, and obtaining the corresponding boarding station according to the timestamp of the boarding movement trajectory video stream of each boarding passenger, and storing the UUID, facial features, and boarding station corresponding to each boarding passenger in the first linked list; S50, extracting facial features from the front face image of each passenger getting off the bus, and obtaining the corresponding getting off station according to the timestamp of the video stream of the getting off movement trajectory of each passenger getting off the bus, and storing the facial features and the getting off station corresponding to each passenger getting off the bus in a second linked list; S60, matching the facial features in the first linked list with the facial features in the second linked list, and when the match is successful and the two facial features correspond to different boarding and alighting stations, it is recorded as one passenger traffic volume, until all facial features are matched and the total passenger traffic volume is obtained; The step S30 specifically includes the following steps: S301, identifying and comparing the converted frontal face in the face region of the current frame image with the converted frontal face in the face region of the previous frame image; S302, if the compared frontal faces are of the same person, the position of the face region in the current frame image is associated with the position of the face region in the previous frame image and recorded; if the compared frontal faces are not of the same person, the new frontal face is tracked and identified again directly from the current frame; S303, repeating steps S301 to S302 to obtain a moving trajectory of the face area of ​​the same person; S304, marking a get-off determination line at a position close to the door in the image of the video stream, and marking a get-on determination line inside the get-off determination line; S305, determining the passenger as a boarding passenger or an alighting passenger according to the position change of the movement trajectory of the same person relative to the boarding determination line and the alighting determination line; The step S305 specifically includes: When the moving trajectory of the passenger first crosses the getting-off determination line and then crosses the getting-on determination line, the passenger is classified as a getting-on passenger; When the moving trajectory of the passenger first crosses the boarding determination line and then crosses the getting off determination line, the passenger is classified as an getting off passenger.

2. The method according to claim 1, characterized in that In step S20, the face in the marked area is converted into a frontal face by a multi-view frontalization method, which specifically includes the following steps: S2011, obtaining facial images of the same passenger taken at different angles by two cameras at the same time; S2012, rotating the face leaning to the right in the face image to the left and the face leaning to the left to the right through affine transformation, to obtain two frontal face images respectively; S2013, averaging the frontal faces in the two frontal face images to obtain a corrected frontal face.

3. The method according to claim 2, characterized in that In the step S2012, the frontal face conversion is performed by using a stretching formula and a compression formula when rotating the face image to the left or right through affine transformation; Assume that the pixel size of the original image before affine transformation is m×n, (x, y) represents a pixel point on the image; The stretching formula is: The compression formula is: Among them, θ represents the angle of facial rotation, α represents half of the angle between the two cheeks of the face, α∈[70°,85°], θ<70°, nose represents the horizontal pixel coordinates of the position of the nose tip in the original image, m represents the number of pixels in the width direction of the original image, n represents the number of pixels in the height direction of the original image, δ is the relaxation factor, and δ takes a value between 0 and 1.

4. The method according to claim 1, characterized in that: In step S20, the face in the marked area is converted into a frontal face by using a single-side face deep learning frontal face conversion method, which specifically includes the following steps: S2021, collecting video stream images of passengers getting on and off the bus at the door position, where the video stream image of each passenger includes multiple facial gestures; S2022, extracting all facial images from the video stream, grouping the facial images according to different passengers, and marking the facial regions and the front and side faces in the grouped images as a training data set; S2023, using a conditional generative adversarial network model and using the training data set for training; S2024, using the trained model, performs frontal face prediction on the facial images of people getting on and off the vehicle collected during actual vehicle operation, and outputs the frontal face images.

5. The method according to claim 1, characterized in that In step S60, when matching the facial features in the first linked list with the facial features in the second linked list, the similarity between the two facial features to be matched is calculated, and when the calculated similarity is greater than a preset threshold, it indicates that the match is successful.

6. The method according to claim 1, characterized in that In the steps S40 and S50, obtaining the boarding and alighting sites specifically includes: S71, obtaining the timestamp of the video stream of the passenger's boarding trajectory or obtaining the timestamp of the video stream of the passenger's alighting trajectory; S72, obtaining the station announcement information from the bus station announcement device or from the bus management network platform, and matching the time in the station announcement information with the timestamp to obtain the passenger's boarding station or alighting station.

7. A system, characterized in that: The system further comprises a memory, a processor, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

Images