Gate control method and device, equipment and storage medium

Abstract

The invention provides a gate control method and device, equipment and a storage medium. The method comprises the steps that multiple frames of images in a gate area are collected; the gate area comprises a verification area, a gate passing area and an in-station detection area, and the gate passing area is located between the verification area and the in-station detection area; performing pedestrian tracking based on the multi-frame image to obtain a plurality of pedestrian tracking frames; for each pedestrian tracking frame, based on a first position and a second position of the pedestrian tracking frame and a boundary position of the gate passing area and other areas, determining a position of a pedestrian corresponding to the pedestrian tracking frame, the first position being a diagonal position of the second position; the other areas comprise a verification area and an in-station detection area; and on the basis of the position of the pedestrian, when it is determined that the pedestrian moves from the verification area to the in-station detection area and it is determined that no pedestrian exists in the gate door influence area, the gate is controlled to be closed, and the gate door influence area belongs to the sub-area of the gate passing area. According to the invention, the judgment accuracy of pedestrian gate passing can be improved.

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to a gate control method, device, equipment and storage medium. Background Art

[0002] Urban rail transit such as subways and light rails has become an important way for citizens to travel. During peak hours of entering and leaving the station, the personnel passage efficiency and passage judgment accuracy of the gate machine affect people's passage experience.

[0003] In the prior art, multiple groups of infrared radiating tubes are arranged in the gate, which emit infrared rays to detect whether there are pedestrians in the gate passage area. When it is determined that there are pedestrians, the gate will not be closed to ensure the safety of pedestrians.

[0004] However, since there is a certain distance between the multiple groups of infrared radiating tubes installed in the gate, when a pedestrian is between two adjacent groups of infrared radiating tubes, there is a risk of false detection, which reduces the accuracy of identifying the passage of people. Summary of the invention

[0005] The present invention provides a gate control method, device, equipment and storage medium, which are used to solve the defect of low judgment accuracy of pedestrian gate passage in the prior art and improve the judgment accuracy of pedestrian gate passage.

[0006] The present invention provides a gate control method, comprising: Collect multiple frames of images in the gate area; the gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; Perform pedestrian tracking based on the multiple frames of images to obtain multiple pedestrian tracking frames; For each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate passage area and other areas, the position of the pedestrian corresponding to the pedestrian tracking frame is determined, wherein the first position is the diagonal position of the second position; the other areas include the verification area and the in-station detection area; When it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the detection area within the station and that there are no pedestrians in the gate influence area, the gate is controlled to close, and the gate influence area belongs to a sub-area of ​​the gate passage area.

[0007] According to a gate control method provided by the present invention, the first position is the upper left corner position of the pedestrian tracking frame, and the second position is the lower right corner position of the pedestrian tracking frame; The determining the position of the pedestrian corresponding to the pedestrian tracking frame based on the first position and the second position of the pedestrian tracking frame and the boundary position coordinates between the gate passage area and other areas includes: When the average value of the ordinate of the first position and the ordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area and the first condition is met, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the gate passage area; The first condition includes any of the following: The first horizontal coordinate of the boundary position between the gate passage area and the verification area is located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The second horizontal coordinate of the boundary position between the gate passage area and the in-station detection area is located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The first horizontal coordinate and the second horizontal coordinate are located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The abscissa of the first position and the abscissa of the second position are located between the first abscissa and the second abscissa.

[0008] According to a gate control method provided by the present invention, the method further includes: When the average value of the vertical coordinate of the first position and the vertical coordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area, and the horizontal coordinate of the second position is greater than the second horizontal coordinate, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the detection area within the station.

[0009] According to a gate control method provided by the present invention, the method further includes: Detecting the number of target pedestrians in the gate passage area; When the number of pedestrians is greater than the number of authorized pedestrians, the height of each main pedestrian among the target pedestrians is obtained; the main pedestrian is a pedestrian walking independently; Based on the heights of the main pedestrians, determining the number of target main pedestrians whose heights are greater than a preset height; When the number of the target pedestrians is greater than the number of the authorized pedestrians, an alarm message is output.

[0010] According to a gate control method provided by the present invention, the step of obtaining the height of each main pedestrian among the target pedestrians includes: Collecting a depth image corresponding to each frame of the image in the gate area; For each frame of the image, align the image with the corresponding depth image to obtain depth information of each pixel in the image; Determining the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixels; Based on the depth information of the top feature point, the height of the main pedestrian in the image is determined.

[0011] According to a gate control method provided by the present invention, the step of determining the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixel points includes: Input each of the images into a pedestrian detection model in sequence, and obtain the detection frame of the main pedestrian in the pedestrian detection frame output by the pedestrian detection model and the position information of the feature point on the top of the head of the main pedestrian; the pedestrian detection model is obtained by training an initial pedestrian detection model based on the sample image and the sample main pedestrian detection frame label and the position label of the feature point on the top of the head of the sample main pedestrian in the sample image; Based on the detection frame of the main pedestrian and the position information of the top feature point of the main pedestrian, the depth information of the top feature point of the main pedestrian in the image is determined from the depth information of all pixels.

[0012] According to a gate control method provided by the present invention, the method further includes: For each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate influence area and other areas, determining the target position of the pedestrian corresponding to the pedestrian tracking frame; When it is determined based on each of the target positions that there are no pedestrians in the gate influence area and the received authorization application is approved, the gate is controlled to open.

[0013] The present invention also provides a gate control device, comprising: An acquisition module, used for acquiring multiple frames of images in the gate area; the gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; A tracking module, used for performing pedestrian tracking based on the multiple frames of images to obtain multiple pedestrian tracking frames; A determination module, for determining, for each of the pedestrian tracking frames, a position of a pedestrian corresponding to the pedestrian tracking frame based on a first position and a second position of the pedestrian tracking frame and a boundary position between the gate passage area and other areas, wherein the first position is a diagonal position of the second position; and the other areas include the verification area and the in-station detection area; The control module is used to control the gate to close when it is determined that the pedestrian has moved from the verification area to the detection area within the station based on the position of the pedestrian and there are no pedestrians in the gate influence area, and the gate influence area belongs to a sub-area of ​​the gate passage area.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any gate control method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the gate control method as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the gate control method as described above is implemented.

[0017] The gate control method, device, equipment and storage medium provided by the present invention collect multiple frames of images in the gate area, and track pedestrians based on the multiple frames of images to obtain multiple pedestrian tracking frames. For each pedestrian tracking frame, the position of the pedestrian corresponding to the pedestrian tracking frame can be determined based on the first position and the second position of the pedestrian tracking frame and the boundary position between the gate passage area and other areas, wherein the first position and the second position are diagonal positions. When it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the station detection area and that there are no pedestrians in the gate influence area, the gate is controlled to close. Since the diagonal position of the pedestrian tracking frame is compared with the boundary position between the gate passage area and other areas, the current passage area of ​​the pedestrian can be accurately determined, thereby judging whether the pedestrian has passed through the gate passage area, and improving the accuracy of pedestrian gate passage judgment. In addition, when it is determined that a pedestrian has moved from the verification area to the inspection area within the station, the gate is controlled to close, which can effectively avoid the safety hazards caused by closing the gate door before the pedestrian leaves the gate passage area, thereby reducing the probability of accidents among passers-by. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic flow chart of a gate control method provided in an embodiment of the present invention.

[0020] Figure 2 A schematic diagram of a gate area provided in an embodiment of the present invention.

[0021] Figure 3 A schematic diagram of the structure of a gate control device provided in an embodiment of the present invention.

[0022] Figure 4 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] At present, in urban rail transit such as subways or light rails, in order to ensure the personal safety of pedestrians, the gate door will only be closed when the pedestrians completely pass through the gate passage area. Therefore, it is very important to accurately judge whether the pedestrian has passed the gate passage area from the verification area and moved from the gate passage area to the detection area in the station. At present, multiple groups of infrared radiators are usually set in the gate, and these infrared radiators detect whether the pedestrian has passed through the gate passage area by emitting infrared rays. However, in this method, since there is a certain distance between the multiple groups of infrared radiators set in the gate, when the pedestrian is between two adjacent groups of infrared radiators, there will be a false detection situation where the pedestrian has not walked out of the gate passage area, but the infrared radiator cannot detect the pedestrian, causing the gate of the gate to close, which not only leads to low accuracy in identifying the passage of personnel, but also increases the risk of accidents for pedestrians.

[0025] In view of the above problems, an embodiment of the present invention provides a gate control method, in which an infrared radiator is not provided in the gate, that is, whether a pedestrian has passed through the gate passage area is not determined by infrared means, but by collecting multiple frames of images in the gate area, tracking pedestrians in the multiple frames of images, and obtaining the pedestrian tracking frame corresponding to each pedestrian, the position of the pedestrian corresponding to the pedestrian tracking frame is determined based on the positions of the two diagonal corners of the pedestrian tracking frame and the boundary position between the gate passage area and other areas, so as to determine whether the pedestrian has moved from the verification area to the station detection area, and when it is determined that the pedestrian has passed through the gate passage area and entered the station detection area, and there are no pedestrians in the gate door influence area, the gate is controlled to be closed. Since the diagonal position of the pedestrian tracking frame is compared with the boundary position between the gate passage area and other areas, the current passage area of ​​the pedestrian can be accurately determined, so as to determine whether the pedestrian has passed through the gate passage area, thereby improving the accuracy and safety of pedestrian gate passage judgment. In addition, in the embodiment of the present invention, since the pedestrian passage can be determined by image detection, there is no need to set up multiple groups of infrared emitting tubes in the gate, so the size of the gate can be effectively reduced.

[0026] Combine the following Figure 1 or Figure 2 The gate control method provided by the embodiment of the present invention is described. The embodiment of the present invention can be applicable to the gate control scenario in rail transit. The execution subject of the method can be a gate or a device for controlling a gate, such as a computer, a server, a server cluster or a specially designed gate control device or other electronic device, or a gate control device provided in the electronic device, which can be implemented by software, hardware or a combination of the two.

[0027] Figure 1 A schematic diagram of a flow chart of a gate control method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes: Step 101: Capture multiple frames of images in the gate area; the gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area.

[0028] In this step, the verification area can be understood as the area where pedestrians undergo identity verification, and can also be called a card swiping area, such as the area for swiping a QR code, swiping a ticket, or swiping a vein palm print, etc. The gate passage area can be the physical passage area that pedestrians need to pass through when entering or exiting the station after passing the verification. Gate doors, such as leaf doors, swing doors, or three-roller gates, are set in the gate passage area. The in-station detection area can be the area in the station that pedestrians enter after passing through the gate.

[0029] An image acquisition device may be installed on the top of the gate to acquire multiple frames of images in the gate area, wherein the image contains information of the verification area, the gate passage area, and the in-station detection area. In this embodiment, the image acquisition device may be a color camera, and the viewing angle of the color camera adopts a top-down viewing angle, that is, the lens of the color camera faces the ground, and the line connecting the physical center of the color camera lens and the lens focus is perpendicular to the ground.

[0030] Step 102: Pedestrian tracking is performed based on multiple image frames to obtain multiple pedestrian tracking frames.

[0031] In this step, the collected multiple frames of images can be input into the target recognition model to obtain the pedestrian detection frame corresponding to each pedestrian, and the pedestrian detection frame can be input into the tracking module to obtain the tracking information of each pedestrian. The tracking information includes the tracked pedestrian ID and the location information of the pedestrian tracking frame. In this embodiment, the tracking algorithm of the tracking module can adopt the Simple Online and Realtime Tracking (SORT) algorithm, the Simple Online and Realtime Tracking with a deepassociation metric (DeepSORT) algorithm, the BoT-SORT algorithm or the ByteTrack algorithm.

[0032] The pedestrian tracking frame obtained by tracking the pedestrian can be updated as the pedestrian moves.

[0033] Step 103: For each of the pedestrian tracking frames, based on the first position, the second position of each pedestrian tracking frame, and the boundary position between the gate passage area and other areas, determine the position of the pedestrian corresponding to the pedestrian tracking frame, where the first position is the diagonal position of the second position; other areas include the verification area and the in-station detection area.

[0034] In this step, the position coordinates of the first position and the second position of the pedestrian tracking frame can be used to represent the position of the pedestrian tracking frame, wherein the first position and the second position are two diagonals of the pedestrian tracking frame, such as the first position is the upper left corner of the pedestrian tracking frame, and the second position is the lower right corner of the pedestrian tracking frame, or the first position is the lower right corner of the pedestrian tracking frame, and the first position is the upper left corner of the pedestrian tracking frame.

[0035] Since the first position and the second position can represent the position of the pedestrian tracking frame, and the position of the pedestrian tracking frame is the position of the pedestrian, the position coordinates of the first position and the second position of each pedestrian tracking frame can be compared with the position coordinates of the boundary position between the gate passage area and other areas to determine the passage position of the pedestrian tracking frame, thereby determining the current passage position of the pedestrian, and based on the passage position, it can be determined whether the pedestrian has moved from the verification area through the gate passage area to the detection area within the station.

[0036] Step 104: When it is determined based on the location of the pedestrian that the pedestrian has moved from the verification area to the detection area within the station and that there are no pedestrians in the gate influence area, the gate is controlled to close, and the gate influence area belongs to a sub-area of ​​the gate passage area.

[0037] In this step, when it is determined that the pedestrian has moved from the verification area to the detection area in the station based on the position of the pedestrian corresponding to the pedestrian tracking frame, it means that the pedestrian has passed the gate passage area. In addition, it is also necessary to determine whether there are pedestrians in the gate influence area. When there are no pedestrians in the gate influence area, the gate will be controlled to close. Among them, the gate influence area belongs to the sub-area of ​​the gate passage area, and can also be understood as the physical area affected by the gate opening and closing. When determining whether there are pedestrians in the gate influence area, it can be determined based on the first position and second position of the pedestrian tracking frame of the pedestrian and the boundary position between the gate influence area and other areas.

[0038] In addition, after the control gate is closed, if it is detected that there are other authorized pedestrians in the verification area, it means that the next pedestrian has swiped the card, and the other authorized pedestrian can be allowed to pass. At this time, the control gate door will be opened again.

[0039] The gate control method provided by the embodiment of the present invention collects multiple frames of images in the gate area, and tracks pedestrians based on the multiple frames of images. After obtaining multiple pedestrian tracking frames, for each pedestrian tracking frame, the position of the pedestrian corresponding to the pedestrian tracking frame can be determined based on the first position, the second position of the pedestrian tracking frame, and the boundary position between the gate passage area and other areas, wherein the first position and the second position are diagonal positions, and when it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the station detection area, and it is determined that there are no pedestrians in the gate door influence area, the gate is controlled to be closed. Since the diagonal position of the pedestrian tracking frame and the boundary position between the gate passage area and other areas are compared, the current passage area of ​​the pedestrian can be accurately determined, thereby judging whether the pedestrian has passed the gate passage area, and improving the accuracy of pedestrian gate passage judgment. In addition, when it is judged that the pedestrian has moved from the verification area to the station detection area, the gate is controlled to be closed, which can effectively avoid the safety hazard caused by closing the gate door before the pedestrian leaves the gate passage area, thereby reducing the probability of accidents for passers-by.

[0040] Exemplarily, based on the above embodiment, the first position is the upper left corner position of the pedestrian tracking frame, and the second position is the lower right corner position of the pedestrian tracking frame. Based on the first position and the second position of each pedestrian tracking frame, and the boundary position between the gate passage area and other areas, the position of the pedestrian corresponding to the pedestrian tracking frame can be determined in the following manner: When the average value of the ordinate of the first position and the ordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area and the first condition is met, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the gate passage area. The first condition includes any of the following: the first horizontal coordinate of the boundary position between the gate passage area and the verification area is between the horizontal coordinate of the first position and the horizontal coordinate of the second position; the second horizontal coordinate of the boundary position between the gate passage area and the in-station detection area is between the horizontal coordinate of the first position and the horizontal coordinate of the second position; the first horizontal coordinate and the second horizontal coordinate are between the horizontal coordinate of the first position and the horizontal coordinate of the second position; the horizontal coordinate of the first position and the horizontal coordinate of the second position are between the first horizontal coordinate and the second horizontal coordinate.

[0041] Specifically, Figure 2 A schematic diagram of a gate area provided by an embodiment of the present invention, such as Figure 2 As shown in the figure, the gate area includes the verification area, the gate passage area and the station detection area. Assuming that the pedestrian is authenticated or authorized in the verification area, the gate door in the gate passage area opens, and the pedestrian enters the gate passage area from the verification area, and then passes through the gate passage area to enter the station detection area. When the pedestrian enters the station detection area and there are no pedestrians in the gate door influence area, the gate will be controlled to close. After collecting multiple frames of images in the gate area, the above-mentioned areas in the gate area will be marked in each frame of the image.

[0042] When pedestrians are detected in the captured image, different pedestrians can be identified based on the pedestrian tracking ID. For a pedestrian tracking frame corresponding to a pedestrian, the upper left corner of the pedestrian tracking frame is the first position. , the lower right corner is the second position The position of the pedestrian tracking box can be represented by the position coordinates of the first position and the position coordinates of the second position.

[0043] When determining whether the pedestrian corresponding to the pedestrian tracking frame is in the gate passage area, it can be determined whether the pedestrian tracking frame satisfies And the first condition is met, the first condition includes: or or or ,in, Indicates the first edge of the gate passage area, Indicates the second edge of the gate passage area. and The width coordinate range of the gate passage area is between Less than .like Figure 2 As shown, the directions indicated by the vertical coordinates in the first position and the second position and the direction indicated by the width coordinate range of the gate passage area are both perpendicular to the pedestrian passage direction.

[0044] in addition, The first horizontal coordinate indicating the boundary between the gate passage area and the verification area, The second horizontal coordinate representing the boundary between the gate passage area and the detection area within the station.

[0045] After confirming that the traveler tracking frame meets And when the above-mentioned first condition is met, it means that the pedestrian corresponding to the pedestrian tracking frame is in the gate passage area, otherwise, it means that the pedestrian corresponding to the pedestrian tracking frame is not in the gate passage area.

[0046] Exemplarily, whether a pedestrian is in the gate passage area can also be determined in the following manner: judging whether the following conditions are met: , and the first condition, where Indicates the first edge of the gate, Indicates the second edge of the gate, and The range between is the width coordinate range of the gate, where: Less than , is a coefficient greater than 0 and less than 1, is a coefficient greater than 1.

[0047] In this embodiment, the position coordinates of the first position and the second position of the pedestrian tracking frame can be compared with the position coordinates of the boundary position between the gate passage area and other areas to determine the current passage position of the pedestrian corresponding to the pedestrian tracking frame, for example, to determine whether the pedestrian is in the gate passage area. When the pedestrian position is determined in this way, it can be determined only by the position coordinates of the pedestrian tracking frame and the position coordinates of the boundary position between the gate passage area and other areas. Not only can the determination efficiency and accuracy be improved, but there is no need to set up multiple sets of infrared radiating tubes in the gate, which can greatly reduce the size of the gate.

[0048] Exemplarily, based on the above embodiments, when the average value of the vertical coordinate of the first position and the vertical coordinate of the second position of the pedestrian tracking frame are within the width coordinate range of the gate passage area, and the horizontal coordinate of the second position is greater than the second horizontal coordinate, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the detection area within the station.

[0049] Specifically, for each pedestrian tracking frame corresponding to the pedestrian or tracking object, set its initial state vector to [1, 0, 0], indicating that the pedestrian is currently in the verification area and has not yet entered the gate passage area. and , its state vector will be updated to [1, 1, 0], indicating that the pedestrian is currently in the gate passage area. When the coordinates of the pedestrian tracking box meet and , the state changes to [1,1,1], indicating that the pedestrian has left the gate passage area and entered the station detection area. At this time, a gate door closing signal will be sent to the gate door control module to control the gate to close.

[0050] In this embodiment, after comparing the average value of the vertical coordinate of the first position and the vertical coordinate of the second position of the pedestrian tracking frame with the width coordinate range of the gate passage area, and comparing the horizontal coordinate of the second position with the second horizontal coordinate of the boundary position of the gate passage area and the station detection area, if it is determined that the pedestrian corresponding to the pedestrian tracking frame has left the gate passage area and entered the station detection area, the gate is controlled to close, which can ensure that the gate door is closed when there are no pedestrians in the gate passage area, reducing the risk of accidents involving pedestrians.

[0051] Exemplarily, on the basis of the above-mentioned embodiments, the number of target pedestrians in the gate passage area can also be detected. When the number of pedestrians is greater than the number of authorized pedestrians, the height of each main pedestrian among the target pedestrians is obtained; the main pedestrians are pedestrians walking independently; based on the height of each main pedestrian, the number of target main pedestrians whose height is greater than the preset height is determined; when the number of target main pedestrians is greater than the number of authorized pedestrians, an alarm message is output.

[0052] Specifically, after the pedestrian applies for authorization and the application is approved and the gate is controlled to open, the pedestrian will walk through the gate passage area to the detection area in the station. In actual scenarios, there may be pedestrians with children or other people who use the card to swipe the card, that is, after authorizing one person, multiple users pass through the gate. In order to prevent swipe card behavior, the embodiment of the present invention can detect the number of target pedestrians in the gate passage area. Exemplarily, the number of pedestrians can be obtained by image detection.

[0053] When the number of pedestrians is greater than the number of authorized pedestrians, it means that the number of pedestrians passing through the gate is greater than the number of pedestrians currently undergoing identity authentication or swiping cards, and it is necessary to obtain the height of each main pedestrian among all target pedestrians passing through the gate passage area. Among them, the main pedestrian is a pedestrian who can walk independently. For example, when a pedestrian is holding a child, the child is not a main pedestrian.

[0054] After obtaining the height of each main pedestrian, the number of target main pedestrians whose height is greater than a preset height can be determined based on the height of each main pedestrian, where the preset height can be 1.2 meters, for example. In practical applications, in rail transit, pedestrians whose height is less than the preset height can be exempted from tickets. Therefore, when determining whether there is free-riding behavior, it is necessary to determine the number of target main pedestrians whose height is greater than the preset height.

[0055] When the number of target pedestrians whose height is greater than the preset height is greater than the number of authorized pedestrians, it means that there are target pedestrians who exceed the preset height and have not undergone identity authentication or swiped their cards to enter the gate passage area, and an alarm message will be output. This alarm message is used to remind the staff that there are pedestrians following or breaking through the gate.

[0056] When the number of target pedestrians whose height is greater than the preset height is equal to the number of authorized pedestrians, it means that the target pedestrians who do not exceed the preset height have not undergone identity authentication or have swiped their cards to enter the gate passage area, and no alarm information will be output.

[0057] In this embodiment, when the number of pedestrians passing through the gate passage area is greater than the number of authorized pedestrians, the height of each main pedestrian passing through the gate passage area can be obtained, and when the number of target main pedestrians with heights greater than the preset height is greater than the number of authorized pedestrians, an alarm message will be output. By determining the height of the main pedestrian, pedestrians who are following or breaking through the gate can be accurately screened out, which improves the traffic efficiency and effectively prevents fare evasion.

[0058] Exemplarily, based on the above-mentioned embodiments, when obtaining the height of each main pedestrian among the target pedestrians, a depth image corresponding to each frame image in the gate area can be collected; for each frame image, the image and the corresponding depth image are aligned to obtain the depth information of each pixel in the image; the depth information of the top feature point of the main pedestrian in the image is determined from the depth information of all pixel points; based on the depth information of the top feature point, the height of the main pedestrian in the image is determined.

[0059] Specifically, in an embodiment of the present invention, in addition to a color camera installed on the top of the gate, a depth camera is also installed. The depth camera can be used to obtain a depth image within the gate area, wherein the color camera and the depth camera can obtain images synchronously, that is, each frame image and the corresponding depth image are images collected at the same time.

[0060] The depth camera can be, for example, a monocular or binocular structured light camera, or a binocular ordinary parallax depth camera or a time of flight (TOF) depth camera. Similar to a color camera, the depth camera also uses a top-down perspective, that is, the lens of the depth camera faces the ground, and the line connecting the physical center of the lens of the depth camera and the lens focus is perpendicular to the ground.

[0061] In addition, the image captured by the color camera can be input into the target detection model to obtain the pedestrian detection frame output by the target detection model and the human head feature points corresponding to the pedestrian detection frame.

[0062] For each frame of color image captured by the color camera, the image can be aligned with the depth image taken at the same time to obtain the depth information of each pixel in the image. Specifically, the intrinsic and extrinsic parameters of the color camera and the depth camera can be obtained by the checkerboard calibration method, and the spatial coordinate system coordinates of the color pixel points in the color camera can be obtained. The depth camera space coordinate system coordinates of the depth pixel corresponding to the color pixel are obtained, and the corresponding pixel coordinate system coordinates are obtained through the conversion relationship between the depth camera space coordinate system and the depth pixel coordinate system, so as to obtain the depth information of each pixel in the color image. is the depth camera space coordinate system coordinate, is the color camera space coordinate system, R is the rotation transformation matrix, and T is the translation vector.

[0063] Based on the depth information of all pixels, the pedestrian detection frame, the feature points on the top of the human head and the depth information of each pixel can be matched. Based on the above alignment relationship, the depth information of the feature points on the top of the human head of the main pedestrian in the image can be determined.

[0064] Furthermore, the height information of the main pedestrian in the image can be obtained by subtracting the depth information of the feature point on the top of the human head from the known height of the depth camera.

[0065] In this embodiment, the image and the corresponding depth image can be aligned to obtain the depth information of each pixel in the image. In this way, the depth information of the top feature point of the main pedestrian in the image can be accurately determined, thereby improving the accuracy of the height of the main pedestrian determined based on the depth information of the top feature point of the head.

[0066] Exemplarily, on the basis of the above-mentioned embodiment, when determining the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixel points, each image can be input into the pedestrian detection model in turn to obtain the detection frame of the main pedestrian and the position information of the top feature point of the main pedestrian in the pedestrian detection frame output by the pedestrian detection model; the pedestrian detection model is obtained by training the initial pedestrian detection model based on the sample image and the sample main pedestrian detection frame label and the position label of the top feature point of the sample main pedestrian in the sample image, and determines the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixel points based on the detection frame of the main pedestrian and the position information of the top feature point of the main pedestrian.

[0067] Specifically, the pedestrian detection model can be, for example, a YOLOv8n-pose model, or a modified model that adds feature point prediction based on yolov5. When training the pedestrian detection model and performing data annotation, the pedestrians in the sample image obtained from a top-down perspective, including pedestrians holding children, carrying children on their backs, and carrying backpacks and luggage, can be framed and annotated as a whole, and a pedestrian annotation frame corresponds to the head height feature point of the main pedestrian in the annotation frame, that is, the annotated label information includes the detection frame label of the sample main pedestrian in the sample image and the position label of the head feature point of the sample main pedestrian. The sample image is input into the initial pedestrian detection model to obtain the predicted sample main pedestrian detection frame and the predicted position of the head feature point of the sample main pedestrian output by the initial pedestrian detection model. After determining the loss information based on the above label information and prediction information, the model parameters of the initial pedestrian detection model are adjusted based on the loss information. The above process is repeated until the model converges or the number of repetitions reaches a preset number of times, and the final model is determined as the pedestrian detection model.

[0068] The multi-frame images collected in real time are input into the trained pedestrian detection model, and the detection frame of the main pedestrian in the pedestrian detection frame and the position information of the feature points on the top of the main pedestrian's head are output.

[0069] It should be noted that there is a correspondence between the detection frame output by the pedestrian detection frame and the top feature points of the main pedestrian. When pedestrian tracking is performed based on the detection frame and the pedestrian tracking frame is determined, there will also be a correspondence between the detection frame and the pedestrian tracking frame. Therefore, the pedestrian tracking frame and the top feature points of the main pedestrian can be matched based on the detection frame. After the pedestrian tracking frame matches the top feature points, when the main pedestrian is tracked, the depth information of the top feature points of the main pedestrian in the image can be determined from the depth information of all pixels based on the position information of the top feature points that match the pedestrian tracking frame of the main pedestrian, and the height of the main pedestrian can be determined based on the depth information.

[0070] In this embodiment, when training the pedestrian detection model, the sample main pedestrian detection frame label and the position label of the top feature point of the sample main pedestrian in the sample image will be marked. In this way, the trained pedestrian detection model can accurately detect the main pedestrian and the top feature point of the main pedestrian, thereby improving the accuracy of the detection result. In addition, the target detection model obtained by the above training method can solve the problem that two heads appear at the same time in the detected pedestrian detection frame, and it is impossible to accurately determine which head to use as the height calculation point. In this embodiment, the top feature point of the main pedestrian in the pedestrian detection frame can be determined, and this method is more accurate than directly taking the center point of the head detection frame as the height calculation point.

[0071] Exemplarily, on the basis of the above-mentioned embodiments, for each pedestrian tracking frame, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate influence area and other areas, the target position of the pedestrian corresponding to the pedestrian tracking frame is determined, and based on each target position, when it is determined that there are no pedestrians in the gate influence area and the received authorization application is approved, the gate is controlled to open.

[0072] Specifically, based on the first position, the second position of the pedestrian tracking frame, and the boundary position between the gate influence area and other areas, it can be determined whether the pedestrian is in the gate influence area. The specific determination method is similar to the method for determining whether the pedestrian is in the gate passage area, which will not be repeated here.

[0073] When it is determined that there are no pedestrians in the gate's influence area and the authorization application received at this time is approved, the gate will be controlled to open so that pedestrians can pass normally. This method can avoid accidents and improve traffic safety.

[0074] The gate control device provided by the present invention is described below. The gate control device described below and the gate control method described above can be referenced to each other.

[0075] Figure 3 The schematic diagram of the structure of the gate control device provided by the embodiment of the present invention is shown in FIG. Figure 3 As shown, the gate control device 300 includes: The acquisition module 11 is used to acquire multiple frames of images in the gate area; the gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; A tracking module 12, configured to perform pedestrian tracking based on the multiple frames of images to obtain multiple pedestrian tracking frames; The determination module 13 is used to determine the position of the pedestrian corresponding to each of the pedestrian tracking frames based on the first position and the second position of the pedestrian tracking frame and the boundary position between the gate passage area and other areas, wherein the first position is the diagonal position of the second position; the other areas include the verification area and the in-station detection area; The control module 14 is used to control the gate to close when it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the detection area within the station and that there are no pedestrians in the gate influence area, and the gate influence area belongs to a sub-area of ​​the gate passage area.

[0076] In an exemplary embodiment, the first position is the upper left corner position of the pedestrian tracking frame, and the second position is the lower right corner position of the pedestrian tracking frame; The determination module 13 is specifically used for: When the average value of the ordinate of the first position and the ordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area and the first condition is met, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the gate passage area; The first condition includes any of the following: The first horizontal coordinate of the boundary position between the gate passage area and the verification area is located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The second horizontal coordinate of the boundary position between the gate passage area and the in-station detection area is located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The first horizontal coordinate and the second horizontal coordinate are located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The abscissa of the first position and the abscissa of the second position are located between the first abscissa and the second abscissa.

[0077] In an exemplary embodiment, the control module 14 is further configured to: When the average value of the vertical coordinate of the first position and the vertical coordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area, and the horizontal coordinate of the second position is greater than the second horizontal coordinate, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the detection area within the station.

[0078] In an exemplary embodiment, the apparatus further comprises: a detection module, an acquisition module and an output module, wherein: A detection module, used to detect the number of target pedestrians in the gate passage area; An acquisition module, used for acquiring the height of each main pedestrian among the target pedestrians when the number of the pedestrians is greater than the number of authorized pedestrians; the main pedestrians are pedestrians walking independently; The determination module 13 is further used to determine the number of target main pedestrians whose heights are greater than a preset height based on the heights of the main pedestrians; The output module is used to output an alarm message when the number of the target pedestrians is greater than the number of the authorized pedestrians.

[0079] In an exemplary embodiment, the acquisition module is specifically configured to: Collecting a depth image corresponding to each frame of the image in the gate area; For each frame of the image, align the image with the corresponding depth image to obtain depth information of each pixel in the image; Determining the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixels; Based on the depth information of the top feature point, the height of the main pedestrian in the image is determined.

[0080] In an exemplary embodiment, the determination module 13 is further configured to: Input each of the images into a pedestrian detection model in sequence, and obtain the detection frame of the main pedestrian in the pedestrian detection frame output by the pedestrian detection model and the position information of the feature point on the top of the head of the main pedestrian; the pedestrian detection model is obtained by training an initial pedestrian detection model based on the sample image and the sample main pedestrian detection frame label and the position label of the feature point on the top of the head of the sample main pedestrian in the sample image; Based on the detection frame of the main pedestrian and the position information of the top feature point of the main pedestrian, the depth information of the top feature point of the main pedestrian in the image is determined from the depth information of all pixels.

[0081] In an exemplary embodiment, the determination module 13 is further configured to: For each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate influence area and other areas, determining the target position of the pedestrian corresponding to the pedestrian tracking frame; When it is determined based on each of the target positions that there are no pedestrians in the gate influence area and the received authorization application is approved, the gate is controlled to open.

[0082] The device of this embodiment can be used to execute the method of any embodiment in the gate control method side embodiment. Its specific implementation process and technical effects are similar to those in the gate control method side embodiment. For details, please refer to the detailed introduction in the gate control method side embodiment, which will not be repeated here.

[0083] Figure 4 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 4As shown, the electronic device may include: a processor (processor) 410 , a communication interface (Communications Interface) 420 , a memory (memory) 430 and a communication bus 440 , wherein the processor 410 , the communication interface 420 , and the memory 430 communicate with each other through the communication bus 440 . The processor 410 can call the logic instructions in the memory 430 to execute the gate control method, which includes: collecting multiple frames of images in the gate area; the gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; pedestrian tracking is performed based on the multiple frames of images to obtain multiple pedestrian tracking frames; for each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate passage area and other areas, the position of the pedestrian corresponding to the pedestrian tracking frame is determined, the first position is the diagonal position of the second position; the other areas include the verification area and the in-station detection area; when it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the in-station detection area, and it is determined that there are no pedestrians in the gate door influence area, the gate is controlled to be closed, and the gate door influence area belongs to a sub-area of ​​the gate passage area.

[0084] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0085] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the gate control method provided by the above methods, which includes: collecting multiple frames of images in the gate area; the gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; pedestrian tracking is performed based on the multiple frames of images to obtain multiple pedestrian tracking frames; for each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate passage area and other areas, the position of the pedestrian corresponding to the pedestrian tracking frame is determined, the first position is the diagonal position of the second position; the other areas include the verification area and the in-station detection area; when it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the in-station detection area, and it is determined that there are no pedestrians in the gate door influence area, the gate is controlled to be closed, and the gate door influence area belongs to a sub-area of ​​the gate passage area.

[0086] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the gate control method provided by the above-mentioned methods, the method comprising: acquiring multiple frames of images within a gate area; the gate area comprises a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; performing pedestrian tracking based on the multiple frames of images to obtain multiple pedestrian tracking frames; for each of the pedestrian tracking frames, determining the position of the pedestrian corresponding to the pedestrian tracking frame based on the first position, the second position of the pedestrian tracking frame, and the boundary position between the gate passage area and other areas, the first position is the diagonal position of the second position; the other areas include the verification area and the in-station detection area; when it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the in-station detection area, and it is determined that there are no pedestrians in the gate door influence area, controlling the gate to close, and the gate door influence area belongs to a sub-area of ​​the gate passage area.

[0087] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gate control method, characterized in that: include: Collect multiple frames of images in the gate area; The gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; Perform pedestrian tracking based on the multiple frames of images to obtain multiple pedestrian tracking frames; For each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate passage area and other areas, the position of the pedestrian corresponding to the pedestrian tracking frame is determined, wherein the first position is the diagonal position of the second position; the other areas include the verification area and the in-station detection area; When it is determined based on the position of the pedestrian that the pedestrian has moved from the verification area to the detection area within the station and that there are no pedestrians in the gate influence area, the gate is controlled to close, and the gate influence area belongs to a sub-area of ​​the gate passage area.

2. The gate control method according to claim 1, characterized in that: The first position is the upper left corner position of the pedestrian tracking frame, and the second position is the lower right corner position of the pedestrian tracking frame; The determining the position of the pedestrian corresponding to the pedestrian tracking frame based on the first position and the second position of the pedestrian tracking frame and the boundary position between the gate passage area and other areas includes: When the average value of the ordinate of the first position and the ordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area and the first condition is met, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the gate passage area; The first condition includes any of the following: The first horizontal coordinate of the boundary position between the gate passage area and the verification area is located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The second horizontal coordinate of the boundary position between the gate passage area and the in-station detection area is located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The first horizontal coordinate and the second horizontal coordinate are located between the horizontal coordinate of the first position and the horizontal coordinate of the second position; The abscissa of the first position and the abscissa of the second position are located between the first abscissa and the second abscissa.

3. The gate control method according to claim 2, characterized in that: The method further comprises: When the average value of the vertical coordinate of the first position and the vertical coordinate of the second position of the pedestrian tracking frame is within the width coordinate range of the gate passage area, and the horizontal coordinate of the second position is greater than the second horizontal coordinate, it is determined that the pedestrian corresponding to the pedestrian tracking frame is located in the detection area within the station.

4. The gate control method according to any one of claims 1 to 3, characterized in that: The method further comprises: Detecting the number of target pedestrians in the gate passage area; When the number of pedestrians is greater than the number of authorized pedestrians, the height of each main pedestrian among the target pedestrians is obtained; the main pedestrian is a pedestrian walking independently; Based on the heights of the main pedestrians, determining the number of target main pedestrians whose heights are greater than a preset height; When the number of the target pedestrians is greater than the number of the authorized pedestrians, an alarm message is output.

5. The gate control method according to claim 4, characterized in that: The obtaining the height of each main pedestrian among the target pedestrians includes: Collecting a depth image corresponding to each frame of the image in the gate area; For each frame of the image, align the image with the corresponding depth image to obtain depth information of each pixel in the image; Determining the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixels; Based on the depth information of the top feature point, the height of the main pedestrian in the image is determined.

6. The gate control method according to claim 5, characterized in that: The step of determining the depth information of the top feature point of the main pedestrian in the image from the depth information of all pixels includes: Input each of the images into a pedestrian detection model in sequence, and obtain the detection frame of the main pedestrian in the pedestrian detection frame output by the pedestrian detection model and the position information of the feature point on the top of the head of the main pedestrian; the pedestrian detection model is obtained by training an initial pedestrian detection model based on the sample image and the sample main pedestrian detection frame label and the position label of the feature point on the top of the head of the sample main pedestrian in the sample image; Based on the detection frame of the main pedestrian and the position information of the top feature point of the main pedestrian, the depth information of the top feature point of the main pedestrian in the image is determined from the depth information of all pixels.

7. The gate control method according to claim 1, characterized in that: The method further comprises: For each of the pedestrian tracking frames, based on the first position and the second position of the pedestrian tracking frame, and the boundary position between the gate influence area and other areas, determining the target position of the pedestrian corresponding to the pedestrian tracking frame; When it is determined based on each of the target positions that there are no pedestrians in the gate influence area and the received authorization application is approved, the gate is controlled to open.

8. A gate control device, characterized in that: include: An acquisition module, used to acquire multiple frames of images in the gate area; The gate area includes a verification area, a gate passage area and an in-station detection area, and the gate passage area is located between the verification area and the in-station detection area; A tracking module, used for performing pedestrian tracking based on the multiple frames of images to obtain multiple pedestrian tracking frames; A determination module, for determining, for each of the pedestrian tracking frames, a position of a pedestrian corresponding to the pedestrian tracking frame based on a first position and a second position of the pedestrian tracking frame and a boundary position between the gate passage area and other areas, wherein the first position is a diagonal position of the second position; and the other areas include the verification area and the in-station detection area; The control module is used to control the gate to close when it is determined that the pedestrian has moved from the verification area to the detection area within the station based on the position of the pedestrian and there are no pedestrians in the gate influence area, and the gate influence area belongs to a sub-area of ​​the gate passage area.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the gate control method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the gate control method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the gate control method according to any one of claims 1 to 7 is implemented.

Images