Passenger shuttle bus get-on and get-off detection method and device, computer equipment and storage medium
Through deep learning algorithms and image analysis technology, combined with road environment information, the running status and pedestrian movement trajectory of passenger buses are automatically identified, and the problems of low efficiency and insufficient accuracy of passenger buses illegally boarding and dropping off passengers in the existing technology are solved, and the problem of insufficient efficiency and accuracy of passenger buses is realized. Efficient and automatic identification of violations.
Patent Information
- Application Number
- CN202411285706.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2025-01-07
AI Technical Summary
The prior art has problems such as poor timeliness, a lot of manpower, low recognition efficiency and insufficient accuracy when identifying passenger buses illegally boarding and getting off passengers.
By collecting target images at preset time intervals, using deep learning algorithms to perform target detection and motion trajectory analysis, combining road environment information to determine whether there are illegal passengers and passengers, and obtain vehicle information.
It improves the detection efficiency, accuracy and robustness of passenger buses illegally boarding and dropping passengers, realizes automated video data analysis, and reduces manpower consumption.
Smart Images
Figure CN119274149A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video image processing, and in particular to a method, a device, a computer device and a storage medium for detecting passengers getting on and off a passenger bus. Background Art
[0002] With the improvement of digital acquisition technology, video surveillance systems have become increasingly common, and the number of surveillance cameras installed in cities has also increased dramatically, accumulating a large amount of video data. These video data can provide more comprehensive and continuous monitoring capabilities for traffic violations based on monitoring systems, such as identifying illegal boarding and alighting behaviors of passenger buses based on video data collected by video surveillance.
[0003] At present, the existing technology generally adopts the method of having relevant professionals review the in-vehicle video surveillance or using video processing technology and fusion management technology to achieve identification and judgment when identifying illegal boarding and alighting behaviors of passenger buses. However, both methods have certain defects. For example, when relevant professionals review the in-vehicle video surveillance, there will be problems such as poor timeliness and the need to consume a lot of manpower. For example, when using video processing technology and fusion management technology, it is impossible to fully realize the full intelligent identification of illegal boarding and alighting behaviors, resulting in low identification efficiency and insufficient identification accuracy. Summary of the invention
[0004] The embodiments of the present invention provide a method, device, computer equipment and storage medium for detecting boarding and alighting of passengers on a passenger bus, aiming to improve the detection efficiency and accuracy of illegal boarding and alighting of passengers on a passenger bus.
[0005] In a first aspect, an embodiment of the present invention provides a method for detecting passengers getting on and off a passenger bus, comprising:
[0006] At every preset time interval, a target image including a passenger bus and pedestrians is collected;
[0007] Performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image;
[0008] Based on the result of target detection, the running status of the passenger bus in the target image is obtained;
[0009] When the passenger bus is in a stationary state, performing a motion trajectory analysis on the pedestrian in the target image in combination with the road environment information;
[0010] Based on the results of the operation trajectory analysis, determine whether there is any illegal boarding or disembarking behavior, and obtain the corresponding vehicle information after determining that there is illegal boarding or disembarking behavior.
[0011] In a second aspect, an embodiment of the present invention provides a passenger bus boarding and alighting detection device, comprising:
[0012] An image acquisition unit, used for acquiring target images including passenger buses and pedestrians at preset time intervals;
[0013] A target detection unit, used for performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image;
[0014] A state acquisition unit, used for acquiring the running state of the passenger bus in the target image based on the result of the target detection;
[0015] A trajectory analysis unit, used for analyzing the motion trajectory of the pedestrian in the target image in combination with the road environment information when the passenger bus is in a stationary state;
[0016] The violation judgment unit is used to judge whether there is any illegal boarding and disembarking behavior based on the result of the operation trajectory analysis, and obtain the corresponding vehicle information after judging that there is any illegal boarding and disembarking behavior.
[0017] In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the passenger boarding and disembarking detection method for a passenger bus as described in the first aspect is implemented.
[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the passenger boarding and alighting detection method for a passenger bus as described in the first aspect is implemented.
[0019] The embodiment of the present invention provides a passenger bus boarding and disembarking detection method, device, computer equipment and storage medium, the method comprising: collecting a target image containing a passenger bus and a pedestrian at every preset time interval; performing target detection on the target image to obtain the passenger bus, pedestrian and road environment information in the target image; based on the result of the target detection, obtaining the running state of the passenger bus in the target image; when the running state of the passenger bus is stationary, performing motion trajectory analysis on the pedestrian in the target image in combination with the road environment information; based on the result of the running trajectory analysis, judging whether there is illegal boarding and disembarking behavior, and obtaining the corresponding vehicle information after judging that there is illegal boarding and disembarking behavior. The embodiment of the present invention detects the collected image data through a deep learning algorithm, and analyzes the running state of the passenger bus, the motion trajectory of the pedestrian and the road environment information in the image data, so as to detect and identify whether there is illegal boarding and disembarking of the passenger bus, so as to improve the detection efficiency, detection accuracy and robustness of illegal boarding and disembarking of the passenger bus. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0021] Figure 1 A schematic flow chart of a method for detecting passengers getting on and off a passenger bus provided by an embodiment of the present invention;
[0022] Figure 2 Another schematic diagram of a flow chart of a method for detecting passengers getting on and off a passenger bus provided by an embodiment of the present invention;
[0023] Figure 3 A schematic diagram of a sub-process of a passenger bus boarding and alighting detection method provided by an embodiment of the present invention;
[0024] Figure 4 A schematic block diagram of a passenger boarding and alighting detection device for a passenger bus provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0027] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0028] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0029] See below Figure 1 , a passenger bus boarding and alighting detection method provided by an embodiment of the present invention specifically comprises: steps S101 to S105.
[0030] Step S101, collecting target images including passenger buses and pedestrians at preset time intervals;
[0031] Step S102: performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image;
[0032] Step S103: based on the target detection result, obtaining the running status of the passenger bus in the target image;
[0033] Step S104: when the passenger bus is in a stationary state, a motion trajectory analysis is performed on the pedestrian in the target image in combination with the road environment information;
[0034] Step S105: Based on the result of the running trajectory analysis, determine whether there is any illegal boarding or disembarking behavior, and obtain corresponding vehicle information after determining that there is illegal boarding or disembarking behavior.
[0035] In this embodiment, firstly, multiple image data are continuously collected, and then the collected image data are subjected to target detection through a deep learning algorithm to detect the passenger bus, pedestrians and road environment information in the image, and then the operation of the passenger bus is analyzed to determine whether the passenger bus is in motion or stationary. If the passenger bus is determined to be in motion, since it is impossible to get on and off passengers in motion, there is no need to continue to analyze the pedestrian movement trajectory and road environment information. If the passenger bus is determined to be stationary, the road environment information is continued to be analyzed, that is, when the passenger bus is in the set passenger boarding and disembarking area, even if there is passenger boarding and disembarking, it is not an illegal operation. When the passenger bus is in a non-passenger boarding and disembarking area, if there is passenger boarding and disembarking at this time, it can be identified as illegal passenger boarding and disembarking, so the pedestrian movement trajectory is continued to be analyzed to confirm whether there is passenger boarding and disembarking at this time. When it is detected that the passenger bus has illegal passenger boarding and disembarking, its vehicle information is extracted and saved for subsequent tracing.
[0036] This embodiment detects the collected image data through a deep learning algorithm, and analyzes the operation status of the passenger bus, the movement trajectory of pedestrians and the road environment information in the image data, so as to detect and identify whether there are any illegal boarding and alighting of passengers on the passenger bus. This can improve the detection efficiency, accuracy and robustness of illegal boarding and alighting of passengers on the passenger bus.
[0037] In one embodiment, the performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image includes:
[0038] Using the YOLOv5 algorithm to perform target detection on the target image to obtain pedestrian, vehicle information and road environment information in the target image;
[0039] The candidate vehicles are identified using a RegNet network to obtain a passenger bus in the target image.
[0040] This embodiment combines the YOLOv5 algorithm and the RegNet network to detect and identify the target image to obtain the transport bus, pedestrians and road environment information in the target image. Of course, in some other embodiments, other deep learning algorithms can also be used to detect and identify the target image.
[0041] In the actual application scenario, a RegNet network trained by the COCO dataset is constructed to identify passenger buses and buses. The RegNet network includes: an initialization layer (Stem) to extract the preliminary feature representation of the image; a backbone layer (Body) to further extract deep features of the image; and an output layer (Head) to map the features to the final category. The initialization layer uses a convolution layer and a channel attention convolution module to initialize the feature extraction of the image to provide input for subsequent network layers. The backbone layer consists of four Stage layers, each of which consists of n bottleneck layers (BottleNeck), and each bottleneck layer has two structures, BottleNeck_s1 and BottleNeck_s2; the backbone of BottleNeck_s1 consists of a convolutional layer with a convolution kernel size of 1×1, a convolutional layer with a convolution kernel size of 3×3, and a convolutional layer with a convolution kernel size of 1×1. The short-cut part is not processed, and the input is directly added to the output; BottleN The backbone of eck_s2 consists of a convolutional layer with a convolution kernel size of 1×1, a convolutional layer with a convolution kernel size of 3×3, and a convolutional layer with a convolution kernel size of 1×1. The shortcut part downsamples the input image through a convolution with a convolution kernel size of 1×1 and then adds it to the output of the backbone. The number of bottleneck layers in the four stage layers is 1, 1, 3, and 7 respectively, and the number of BottleNeck_s1 and BottleNeck_s2 is (1, 0), (1, 0), (1, 3), and (1, 6) respectively. The output layer contains a global average pooling layer and a fully connected layer. Finally, the fully connected layer obtains the features containing the target classification in the feature map. The output in the fully connected layer is converted into a probability distribution through the Softmax function, thereby obtaining the confidence of the passenger bus and bus categories. Through the above network structure, the detected candidate vehicles are further identified using the trained RegNet network to distinguish between passenger buses and buses, thereby filtering out images containing passenger buses.
[0042] In one embodiment, the step of obtaining the running status of the passenger bus in the target image based on the target detection result includes:
[0043] Obtaining a first bounding box of the passenger bus;
[0044] Based on the first bounding box, using the ByteTrack algorithm to track the motion trajectory of the passenger bus to obtain the motion trajectory of the passenger bus;
[0045] Based on the motion trajectory of the passenger bus, determining whether a trajectory change value of the passenger bus exceeds a preset pixel threshold;
[0046] If the trajectory change value of the passenger bus exceeds a preset pixel threshold, it is determined that the passenger bus is in motion;
[0047] If the trajectory change value of the passenger bus does not exceed the preset pixel threshold, it is determined that the passenger bus is in a stationary state.
[0048] In this embodiment, when analyzing the operation status of a passenger bus, the bounding box of the passenger bus in the target image is first obtained. bus =(x b ,y b ,x b +w b ,y b +h b ), where x b and b represents the coordinates of the upper left vertex of the passenger bus bounding box, w b and h b Respectively represent the length and width of the passenger bus boundary box. The ByteTrack algorithm is then used to track the target motion trajectory of the passenger bus. Then, the trajectory change of the passenger bus target between adjacent images is determined based on the target motion trajectory of the passenger bus. When the trajectory change is less than or equal to the set pixel threshold, the passenger bus is determined to be in a stationary state; when the trajectory change is greater than the set pixel threshold, the passenger bus is determined to be in motion. The ByteTrack algorithm described in this embodiment is a multi-target tracking algorithm based on the tracking-by-detection paradigm, which aims to solve some challenges in multi-target tracking, such as missed detection and trajectory fragmentation problems. The ByteTrack algorithm achieves multi-target tracking by associating each detection box, rather than just associating high-score detection boxes.
[0049] In one embodiment, when the passenger bus is in a stationary state, performing motion trajectory analysis on pedestrians in the target image in combination with the road environment information includes:
[0050] When the running state of the passenger bus is a stationary state, judging whether the passenger bus is in an area where passengers can get on and off and an area where passengers cannot get on and off based on the road environment information; wherein the road environment information includes platform signs, stop lines and bus stops;
[0051] When it is determined that the passenger bus is in a non-passenger boarding and disembarking area, a motion trajectory analysis is performed on the pedestrian in the target image.
[0052] Before analyzing the movement trajectory of pedestrians, this embodiment determines the area where the passenger bus is located, that is, determines whether the passenger bus is in the set compliant boarding and alighting area. When the passenger bus is determined to be in the set boarding and alighting area, even if there is boarding and alighting at this time, it does not constitute a violation. When the passenger bus is determined to be in a non-boarding and alighting area, if there is boarding and alighting at this time, it constitutes illegal boarding and alighting. In actual application scenarios, the platform, parking line, and bus stop signs in the road environment information can be combined to determine whether it is in a boarding and alighting area or a non-boarding and alighting area.
[0053] In one embodiment, when the passenger bus is in a stationary state, performing motion trajectory analysis on the pedestrian in the target image in combination with the road environment information further includes:
[0054] Obtain a second bounding box of the pedestrian;
[0055] Calculate the overlap degree between the first bounding box and the second bounding box according to the following formula;
[0056]
[0057] Wherein, O represents the degree of overlap, w i and h i denote the width and height of the intersection of the first bounding box and the second bounding box, respectively, b and h b denote the length and width of the first bounding box, respectively, p and h p represent the length and width of the second bounding box respectively.
[0058] The movement trajectory of the pedestrian is determined according to the overlap degree.
[0059] In this embodiment, when analyzing the motion trajectory of a pedestrian, the pedestrian's bounding box is first obtained. person =(x p ,y p ,x p +w p ,y p +h p ), where x p and p Respectively represent the coordinates of the upper left vertex of the pedestrian bounding box, w p and h p Respectively represent the length and width of the passenger bus bounding box. Next, combine the passenger bus bounding box bus =(x b ,y b ,x b +w b ,yb +h b ) is used to calculate the degree of overlap between the bounding box of the passenger bus and the bounding box of the pedestrian. The specific calculation method is shown in the above formula. Then, the possibility of getting on and off the bus is judged based on the degree of overlap between the bounding box of the passenger bus and the bounding box of the pedestrian.
[0060] Specifically, when a pedestrian is on a passenger bus, the bounding boxes of the passenger bus and the pedestrian completely overlap, and the bounding boxes of the passenger bus and the pedestrian also satisfy x p <x b ,y p <y b 、w b <w p And h b <h p , while the value of O is between 0 and 1 and remains unchanged. When the pedestrian has the behavior of getting on and off the bus, the bounding boxes of the passenger bus and the pedestrian will intersect, and the value of O is between 0 and 1 and continues to change. When the pedestrian completes the behavior of getting off the bus or approaches the passenger bus, the bounding boxes of the passenger bus and the pedestrian do not intersect, and the value of O is 0. In other words, by analyzing the degree of overlap of the bounding boxes of the passenger bus and the pedestrian and comparing the coordinates of the bounding boxes of the passenger bus and the pedestrian, the effect of analyzing whether the pedestrian has the behavior of getting on and off the bus can be achieved. For example, when the overlapping state of the bounding box of the passenger bus and the bounding box of the pedestrian changes from complete overlap → partial overlap (changing over time) → non-overlapping or from partial overlap (changing over time) → non-overlapping, then the pedestrian may be in the state of getting off the bus; when the degree of overlap of the bounding box of the passenger bus and the bounding box of the pedestrian changes from non-overlapping → partial overlap (changing over time) → complete overlap or from partial overlap (changing over time) → complete overlap, then the pedestrian may be in the state of getting on the bus.
[0061] In one embodiment, when the passenger bus is in a stationary state, performing motion trajectory analysis on the pedestrian in the target image in combination with the road environment information further includes:
[0062] Obtain a second bounding box of the pedestrian;
[0063] Based on the second bounding box, obtaining the human body posture key point coordinate information of the pedestrian through the OpenPose method; wherein the human body posture key point information includes left hip coordinate information, right hip coordinate information, left knee coordinate information, right knee coordinate information, left ankle coordinate information and right ankle coordinate information;
[0064] The human body posture key point coordinate information is converted into a human body posture key point vector, and the knee bending angle of the pedestrian is calculated according to the following formula;
[0065]
[0066] Where θ represents the knee bending angle, V i_hip_knee V represents the vector from the left hip to the left knee or the vector from the right hip to the right knee. i_knee_ankle Represents the vector from the left knee to the left ankle or the vector from the right knee to the right ankle.
[0067] It is determined whether the pedestrian is getting on or off the vehicle according to the bending angle of the pedestrian's knees.
[0068] In order to further improve the accuracy of passenger boarding and disembarking detection, this embodiment obtains the key point information of the pedestrian's human body posture to more accurately identify the pedestrian's motion state. Specifically, firstly, based on the pedestrian's bounding box, the OpenPose method is used to obtain the key point information of the pedestrian's human body posture, that is, the coordinates of the pedestrian's left hip, right hip, left knee, right knee, left ankle and right ankle in the target image are obtained, and then the obtained coordinates are converted into corresponding vectors, such as V l_hip_knee represents the vector from the left hip to the left knee, V r_hip_knee represents the vector from the right hip to the right knee, V l_knee_ankle represents the vector from the left knee to the left ankle, V r_knee_ankle Represents the vector from the right knee to the right ankle. Finally, the pedestrian's knee bending angle is calculated using the above-mentioned pedestrian's knee bending angle calculation formula, and whether there is a behavior of getting on or off the vehicle is determined based on the pedestrian's knee bending angle. For example, when the value of the pedestrian's knee bending angle θ between adjacent images gradually decreases from about 180°, and the angle θ is less than 135° and greater than 60°, it can be considered that the pedestrian has a behavior of getting on the vehicle; when the value of the pedestrian's knee bending angle θ between adjacent images gradually increases to about 180°, it can be considered that the pedestrian has a behavior of getting off the vehicle.
[0069] In one embodiment, obtaining corresponding vehicle information after determining that there is illegal boarding and disembarking behavior includes:
[0070] Detecting the license plate position information of the passenger bus by using the YOLOv5 algorithm;
[0071] Get the license plate color information through the RegNet network;
[0072] The character information of the license plate is extracted through the CRNN network.
[0073] After detecting a passenger bus with illegal boarding and alighting, this embodiment can first obtain the location information of the license plate through the YOLOv5 network, then obtain the color information of the license plate through the RegNet network, and finally extract the characters of the license plate through the CRNN network, and save the corresponding license plate information and related image data.
[0074] In actual application scenarios, combined with Figure 2 and Figure 3 When performing passenger boarding and alighting detection on a passenger bus, firstly, the monitoring video data is obtained and preprocessed, then the road environment, pedestrians and vehicles in the video data are detected, and the passenger bus is identified, and then it is judged whether the passenger bus is in the area where passengers are allowed to board and alight according to the road environment. If it is not in the area where passengers are allowed to board and alight, the motion trajectories of the passenger bus and the pedestrian are obtained to judge whether the passenger bus is in a stationary state and whether the pedestrian has boarded and alighted the bus, and then it is judged whether there is illegal boarding and alighting. When illegal boarding and alighting is detected, the vehicle information of the corresponding passenger bus is extracted and saved. Here, when judging whether the pedestrian has boarded and alighted the bus, it can be judged in combination with the change of the pedestrian's boundary box and the change of the pedestrian's knee bending angle. Compared with the prior art, the beneficial effect of the embodiment of the present invention is that the embodiment of the present invention can realize automatic analysis of video data only by a computer, so as to judge whether there is illegal boarding and alighting on the passenger bus, which can effectively improve the detection efficiency and accuracy of illegal behavior.
[0075] Figure 4 A schematic block diagram of a passenger bus boarding and alighting detection device 400 provided in an embodiment of the present invention, the device 400 comprises:
[0076] An image acquisition unit 401 is used to acquire a target image including a passenger bus and pedestrians at a preset time interval;
[0077] The target detection unit 402 is used to perform target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image;
[0078] A state acquisition unit 403, used to acquire the running state of the passenger bus in the target image based on the result of the target detection;
[0079] A trajectory analysis unit 404, configured to perform motion trajectory analysis on pedestrians in the target image in combination with the road environment information when the passenger bus is in a stationary state;
[0080] The violation judgment unit 405 is used to judge whether there is any illegal boarding or disembarking behavior based on the result of the operation trajectory analysis, and obtain corresponding vehicle information after judging whether there is any illegal boarding or disembarking behavior.
[0081] In one embodiment, the target detection unit 402 includes:
[0082] A first detection unit is used to perform target detection on the target image using a YOLOv5 algorithm to obtain information about pedestrians, vehicles and road environment in the target image;
[0083] The vehicle identification unit is used to use the RegNet network to perform vehicle identification on the candidate vehicles to obtain the passenger bus in the target image.
[0084] The state acquisition unit 403 includes:
[0085] A first acquiring unit, configured to acquire a first boundary frame of the passenger bus;
[0086] A trajectory tracking unit, configured to track the motion trajectory of the passenger bus using a ByteTrack algorithm based on the first bounding box to obtain the motion trajectory of the passenger bus;
[0087] A trajectory judgment unit, used to judge whether a trajectory change value of the passenger bus exceeds a preset pixel threshold based on the motion trajectory of the passenger bus;
[0088] A motion determination unit, configured to determine that the passenger bus is in motion if a trajectory change value of the passenger bus exceeds a preset pixel threshold;
[0089] The stationary determination unit is used to determine that the passenger bus is in a stationary state if the trajectory change value of the passenger bus does not exceed a preset pixel threshold.
[0090] In one embodiment, the trajectory analysis unit 404 includes:
[0091] An area judgment unit, used for judging whether the passenger bus is in an area where passengers can get on and off and an area where passengers cannot get on and off based on the road environment information when the passenger bus is in a stationary state; wherein the road environment information includes platform signs, stop lines and bus stops;
[0092] The area determination unit is used to analyze the motion trajectory of the pedestrian in the target image when it is determined that the passenger bus is in a non-passenger boarding and disembarking area.
[0093] In one embodiment, the trajectory analysis unit 404 further includes:
[0094] A second acquisition unit, configured to acquire a second bounding box of the pedestrian;
[0095] an overlap calculation unit, configured to calculate the degree of overlap between the first bounding box and the second bounding box according to the following formula;
[0096]
[0097] Wherein, O represents the degree of overlap, w i and h i denote the width and height of the intersection of the first bounding box and the second bounding box, respectively,b and h b denote the length and width of the first bounding box, respectively, p and h p represent the length and width of the second bounding box respectively.
[0098] A trajectory determination unit is used to determine the movement trajectory of the pedestrian according to the overlap degree.
[0099] In one embodiment, the trajectory analysis unit 404 further includes:
[0100] A third acquisition unit, configured to acquire a second bounding box of the pedestrian;
[0101] A posture acquisition unit, configured to acquire the human body posture key point coordinate information of the pedestrian through an OpenPose method based on the second bounding box; wherein the human body posture key point information includes left hip coordinate information, right hip coordinate information, left knee coordinate information, right knee coordinate information, left ankle coordinate information and right ankle coordinate information;
[0102] A bending calculation unit, used to convert the human body posture key point coordinate information into a human body posture key point vector, and calculate the pedestrian's knee bending angle according to the following formula;
[0103]
[0104] Where θ represents the knee bending angle, V i_hip_knee V represents the vector from the left hip to the left knee or the vector from the right hip to the right knee. i_knee_ankle Represents the vector from the left knee to the left ankle or the vector from the right knee to the right ankle.
[0105] The behavior judgment unit is used to judge whether the pedestrian has the behavior of getting on or off the vehicle according to the knee bending angle of the pedestrian.
[0106] In one embodiment, the violation determination unit 405 includes:
[0107] A second detection unit is used to detect the license plate position information of the passenger bus through the YOLOv5 algorithm;
[0108] A color acquisition unit, used to obtain the color information of the license plate through the RegNet network;
[0109] The character extraction unit is used to extract the character information of the license plate through the CRNN network.
[0110] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0111] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0112] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.
[0113] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.
[0114] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. A passenger bus boarding and alighting detection method, characterized in that: include: At every preset time interval, a target image including a passenger bus and pedestrians is collected; Performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image; Based on the result of target detection, the running status of the passenger bus in the target image is obtained; When the passenger bus is in a stationary state, performing a motion trajectory analysis on the pedestrian in the target image in combination with the road environment information; Based on the results of the operation trajectory analysis, determine whether there is any illegal boarding or disembarking behavior, and obtain the corresponding vehicle information after determining that there is illegal boarding or disembarking behavior.
2. The passenger boarding and alighting detection method of a passenger bus according to claim 1, characterized in that: The performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image includes: Using the YOLOv5 algorithm to perform target detection on the target image to obtain pedestrian, vehicle information and road environment information in the target image; The candidate vehicles are identified using a RegNet network to obtain a passenger bus in the target image.
3. The passenger bus boarding and alighting detection method according to claim 1, characterized in that: The step of obtaining the running status of the passenger bus in the target image based on the target detection result includes: Obtaining a first bounding box of the passenger bus; Based on the first bounding box, using the ByteTrack algorithm to track the motion trajectory of the passenger bus to obtain the motion trajectory of the passenger bus; Based on the motion trajectory of the passenger bus, determining whether a trajectory change value of the passenger bus exceeds a preset pixel threshold; If the trajectory change value of the passenger bus exceeds a preset pixel threshold, it is determined that the passenger bus is in motion; If the trajectory change value of the passenger bus does not exceed the preset pixel threshold, it is determined that the passenger bus is in a stationary state.
4. The passenger bus boarding and alighting detection method according to claim 1, characterized in that: When the passenger bus is in a stationary state, performing motion trajectory analysis on the pedestrian in the target image in combination with the road environment information includes: When the running state of the passenger bus is a stationary state, judging whether the passenger bus is in an area where passengers can get on and off and an area where passengers cannot get on and off based on the road environment information; wherein the road environment information includes platform signs, stop lines and bus stops; When it is determined that the passenger bus is in a non-passenger boarding and disembarking area, a motion trajectory analysis is performed on the pedestrian in the target image.
5. The passenger boarding and alighting detection method of a passenger bus according to claim 3, characterized in that: When the passenger bus is in a stationary state, performing motion trajectory analysis on the pedestrian in the target image in combination with the road environment information further includes: Obtain a second bounding box of the pedestrian; Calculate the overlap degree between the first bounding box and the second bounding box according to the following formula; Wherein, O represents the degree of overlap, w i and h i denote the width and height of the intersection of the first bounding box and the second bounding box, respectively, b and h b denote the length and width of the first bounding box, respectively, p and h p represent the length and width of the second bounding box respectively. The movement trajectory of the pedestrian is determined according to the overlap degree.
6. The passenger bus boarding and alighting detection method according to claim 1, characterized in that: When the passenger bus is in a stationary state, performing motion trajectory analysis on the pedestrian in the target image in combination with the road environment information further includes: Obtain a second bounding box of the pedestrian; Based on the second bounding box, obtaining the human body posture key point coordinate information of the pedestrian through the OpenPose method; wherein the human body posture key point information includes left hip coordinate information, right hip coordinate information, left knee coordinate information, right knee coordinate information, left ankle coordinate information and right ankle coordinate information; The human body posture key point coordinate information is converted into a human body posture key point vector, and the knee bending angle of the pedestrian is calculated according to the following formula; Where θ represents the knee bending angle, V i_hip_knee V represents the vector from the left hip to the left knee or the vector from the right hip to the right knee. i_knee_ankle Represents the vector from the left knee to the left ankle or the vector from the right knee to the right ankle. It is determined whether the pedestrian is getting on or off the vehicle according to the bending angle of the pedestrian's knees.
7. The passenger boarding and alighting detection method of a passenger bus according to claim 1, characterized in that: The obtaining of corresponding vehicle information after determining that there is illegal boarding and disembarking behavior includes: Detecting the license plate position information of the passenger bus by using the YOLOv5 algorithm; Get the license plate color information through the RegNet network; The character information of the license plate is extracted through the CRNN network.
8. A passenger bus boarding and alighting detection device, characterized in that: include: An image acquisition unit, used for acquiring target images including passenger buses and pedestrians at preset time intervals; A target detection unit, used for performing target detection on the target image to obtain passenger bus, pedestrian and road environment information in the target image; A state acquisition unit, used for acquiring the running state of the passenger bus in the target image based on the result of the target detection; A trajectory analysis unit, used for analyzing the motion trajectory of the pedestrian in the target image in combination with the road environment information when the passenger bus is in a stationary state; The violation judgment unit is used to judge whether there is any illegal boarding and disembarking behavior based on the result of the operation trajectory analysis, and obtain the corresponding vehicle information after judging that there is any illegal boarding and disembarking behavior.
9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting passengers getting on and off a passenger bus as claimed in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting passengers getting on and off a passenger bus as claimed in any one of claims 1 to 7 is implemented.