Authorized pedestrian determination method and device, equipment and storage medium

Abstract

The invention provides an authorized pedestrian determination method, apparatus and device, and a storage medium. The method comprises the steps of collecting multiple frames of images in a gate area; a gate is arranged in the gate area; performing pedestrian tracking based on the multiple frames of images to obtain at least one pedestrian tracking frame; and for each pedestrian tracking frame, determining that the target pedestrian is an authorized pedestrian under the condition that it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in an authorization sensing area arranged on the gate and the authorization application received at the current moment passes. According to the embodiment of the invention, whether the target pedestrian needing to pass is the authorized pedestrian or not can be accurately determined.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for determining an authorized pedestrian. Background Art

[0002] With the continuous advancement of science and technology and the popularization of intelligent applications, gate systems, as important entrance and exit control equipment, have been widely used in various fields such as public transportation, office buildings or stadiums.

[0003] In the existing pedestrian detection scheme of the gate machine, usually when it is detected that a pedestrian has submitted an authorization application and the authorization application is approved, the gate machine will be controlled to open to allow pedestrians to pass. However, in the above detection scheme, it is not detected which pedestrian has submitted the authorization application, that is, it is impossible to determine the authorized pedestrian. It may happen that pedestrian A swipes the card while pedestrian B passes. Therefore, how to accurately determine the authorized pedestrian is an important prerequisite for preventing fare evasion and ticket freeloading. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for determining authorized pedestrians, which are used to solve the defect that the authorized pedestrians cannot be determined in the prior art, and achieve the purpose of accurately determining whether the target pedestrian who needs to pass is an authorized pedestrian.

[0005] The present invention provides a method for determining an authorized pedestrian, comprising: Collecting multiple frames of images in a gate area; a gate is arranged in the gate area; Perform pedestrian tracking based on the multiple frames of images to obtain at least one pedestrian tracking frame; For each of the pedestrian tracking frames, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate and the authorization application received at the current moment is passed, the target pedestrian is determined to be an authorized pedestrian.

[0006] According to an authorized pedestrian determination method provided by the present invention, the hand position includes the position of a hand feature point, and the hand feature point is used to characterize the hand; The step of determining that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate includes: Acquire the position information of the hand feature points; When the horizontal coordinate in the position information of the hand feature point is greater than the first horizontal coordinate and less than the second horizontal coordinate, and the vertical coordinate in the position information of the hand feature point is greater than the first vertical coordinate and less than the second vertical coordinate, it is determined that the hand of the target pedestrian is located in the authorization sensing area; the first horizontal coordinate is the starting horizontal coordinate of the authorization sensing area, the second horizontal coordinate is the ending horizontal coordinate of the authorization sensing area, the first vertical coordinate is the starting vertical coordinate of the authorization sensing area, and the second vertical coordinate is the ending vertical coordinate of the authorization sensing area.

[0007] According to a method for determining an authorized pedestrian provided by the present invention, the method further includes: When the following detection conditions are met, determining that the target pedestrian is an unauthorized pedestrian; The detection conditions include: After determining that the hand of the target pedestrian is located in the authorization sensing area and no authorization application is received at the current moment; or,

[0008] After determining that the hand of the target pedestrian is located within the authorization sensing area, and the authorization application currently received is not approved; or,

[0009] It is determined that the hand of the target pedestrian is not located in the authorized sensing area.

[0010] According to a method for determining an authorized pedestrian provided by the present invention, the step of obtaining the position information of the hand feature points includes: Each of the images is input into the pedestrian detection model in turn to obtain the detection frame of the target pedestrian output by the pedestrian detection model and the position information of the hand feature points of the target pedestrian in the detection frame; the pedestrian detection model is obtained by training the initial pedestrian detection model based on the sample image, the sample main pedestrian detection frame label in the sample image and the position label of the hand feature points of the sample main pedestrian; the sample main pedestrian includes pedestrians who can walk independently.

[0011] According to a method for determining authorized pedestrians provided by the present invention, the pedestrian detection model is trained based on the following method: Acquire multiple frames of the sample images; Marking the sample main pedestrian detection frame label and the position label of the hand feature point of the sample main pedestrian in each of the sample images; Input each of the sample images into the initial pedestrian detection model to obtain a predicted detection frame of the sample main pedestrian and a predicted position of a hand feature point of the sample main pedestrian output by the initial pedestrian detection model; Determining loss information based on the detection box label, the position label, the predicted detection box, and the predicted position; Based on the loss information, the model parameters of the initial pedestrian detection model are adjusted to obtain the pedestrian detection model.

[0012] According to a method for determining an authorized pedestrian provided by the present invention, the method further includes: Obtaining the height of the target pedestrian; When the height of the target pedestrian is less than or equal to the preset height, the authorization status of the target pedestrian is updated to an authorized pedestrian.

[0013] According to an authorized pedestrian determination method provided by the present invention, the authorized sensing area includes a palm vein sensing area.

[0014] The present invention also provides an authorized pedestrian determination device, comprising: An acquisition module, used to acquire multiple frames of images in the gate area; A tracking module, configured to perform pedestrian tracking based on the multiple frames of images to obtain at least one pedestrian tracking frame; The control module is used to determine, for each of the pedestrian tracking frames, that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate and the authorization application received at the current moment is passed, and determine that the target pedestrian is an authorized pedestrian.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the authorized pedestrian determination method as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the authorized pedestrian determination method as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the authorized pedestrian determination method as described in any one of the above is implemented.

[0018] The authorized pedestrian determination method, device, equipment and storage medium provided by the present invention collect multiple frames of images in the gate area, track pedestrians based on the multiple frames of images, and obtain at least one pedestrian tracking frame. For each pedestrian tracking frame, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is in the authorization sensing area set on the gate, and the authorization application received at the current moment is passed, the target pedestrian is determined to be an authorized pedestrian, wherein a gate is set in the gate area. In the scenario of obtaining authorization by swiping a card, swiping a QR code or through palm vein, the pedestrian will place his hand in the authorization sensing area to perform the corresponding operation to send an authorization application. When it is determined that the hand of the target pedestrian is in the authorization sensing area and the authorization application received at the current moment is passed, it means that the target pedestrian has executed the authorization application, thereby accurately determining that the target pedestrian is an authorized pedestrian, which can effectively prevent fare evasion or ticket freeloading. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of a flow chart of a method for determining an authorized pedestrian provided in an embodiment of the present invention.

[0021] Figure 2 A schematic diagram of a gate area provided in an embodiment of the present invention.

[0022] Figure 3 A schematic diagram of the structure of an authorized pedestrian determination device provided in an embodiment of the present invention.

[0023] Figure 4 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In the gate access system, card-swiping behaviors often occur. After a user obtains the access authorization, other unauthorized users pass through. For example, the later person follows the previous person to swipe the card to pass through the gate, or the later person swipes the card to pass when the previous person swipes the card. These phenomena will not only cause damage to the interests of the authorized users, but also may cause users to be pinched and injured, thus resulting in low security of passage.

[0026] At present, in the scenario of obtaining authorization by swiping a card, scanning a QR code or palm vein, the gate is usually controlled to open when a pedestrian is detected to have applied for authorization and the authorization application is approved. It does not detect which pedestrian has applied for authorization, which means that it is impossible to determine the authorized pedestrian. Therefore, how to accurately determine the authorized pedestrian is an important prerequisite for preventing fare evasion and ticket freeloading.

[0027] Taking the above problems into consideration, an embodiment of the present invention proposes a method for determining authorized pedestrians, which can determine authorized persons in scenarios where authorization is obtained by swiping a card, swiping a QR code, or swiping a palm vein. Specifically, in this method, by collecting multiple frames of images in the gate area and tracking pedestrians based on the multiple frames of images, at least one pedestrian tracking frame is obtained. For each pedestrian tracking frame, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate, it means that the target pedestrian may have made an action to apply for authorization. If it is determined that the authorization application received at the current moment is passed, it means that the target pedestrian is an authorized pedestrian. In the above manner, it is possible to accurately identify whether the target pedestrian who needs to pass in the scenario where authorization is obtained by swiping a card, swiping a QR code, or swiping a palm vein is an authorized pedestrian.

[0028] Combine the following Figure 1 and Figure 2 The authorized pedestrian determination method provided by an embodiment of the present invention is described. The embodiment of the present invention can be applicable to any scenario where a gate system is used as a traffic control device, such as rail transit, office buildings, libraries, etc. The execution subject of the method can be a gate or a device for controlling a gate, such as a computer, a server, a server cluster, or a specially designed gate control device and other electronic devices, or it can be an authorized pedestrian determination device set in the electronic device, and the authorized pedestrian determination device can be implemented by software, hardware, or a combination of both.

[0029] Figure 1 A flowchart of a method for determining an authorized pedestrian provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes: Step 101: Capture multiple frames of images within the gate area.

[0030] In this step, an image acquisition device may be installed on the top of the gate. In this embodiment, the image acquisition device may be a color camera. The viewing angle of the color camera adopts a top-down viewing angle, that is, the lens of the color camera faces the ground, and the line connecting the physical center of the lens of the color camera and the lens focus is perpendicular to the ground. The image acquisition device can be used to capture a video of each pedestrian who needs to pass through the gate, wherein the multiple frames of images in the video are all color images.

[0031] It should be understood that the above-mentioned image acquisition device can also be other types of cameras, such as black and white cameras, infrared cameras or 3D depth cameras, so as to collect different types of image data, such as black and white images, infrared images or images with depth information, as long as pedestrian tracking can be performed based on the image.

[0032] Step 102: Pedestrian tracking is performed based on multiple frames of images to obtain at least one pedestrian tracking frame.

[0033] In this step, each frame of the collected image can be sequentially input into the target recognition model to obtain a pedestrian detection frame corresponding to the target pedestrian, and the pedestrian detection frame is input into the tracking module to obtain the tracking information of the target pedestrian. The tracking information includes the ID of the target pedestrian and the location information of the pedestrian tracking frame. In this embodiment, the tracking algorithm of the tracking module can adopt a Simple Online and Realtime Tracking (SORT) algorithm, a Simple Online and Realtime Tracking with adeep association metric (DeepSORT) algorithm, a BoT-SORT algorithm, or a ByteTrack algorithm.

[0034] Among them, the pedestrian tracking frame obtained by tracking the target pedestrian will be updated as the pedestrian moves. Therefore, the position information of the pedestrian tracking frame can represent the position of the target pedestrian. There is a corresponding relationship between the pedestrian tracking frame and the head feature points, left hand feature points and right hand feature points of the corresponding pedestrian.

[0035] Step 103: For each pedestrian tracking frame, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate and the authorization application received at the current moment is passed, the target pedestrian is determined to be an authorized pedestrian.

[0036] In this step, Figure 2 A schematic diagram of a gate area provided by an embodiment of the present invention, such as Figure 2As shown, the gate area includes a verification area and a gate passage area, wherein the verification area can be understood as a pedestrian being able to perform verification operations such as swiping a card, swiping a QR code or swiping a vein palm print in the authorization sensing area when in the area, or it can also be understood as an area where pedestrians are waiting to enter the gate passage area. The gate passage area can be a physical passage area that pedestrians need to pass through when entering or exiting the station after passing the verification, and a gate door, such as a leaf door, a swing door or a three-roller gate, is arranged in the gate passage area. When the pedestrian obtains authorization, he will enter the gate passage area from the verification area. The authorization sensing area is an area located on the gate and used to receive the card swiping information or vein palm print information input by the user. It should be understood that the verification area and the gate passage area are physical areas that users pass through when walking, and the authorization sensing area is the area that receives the authorization application.

[0037] In scenarios where authorization is obtained by swiping a card, scanning a QR code, or through palm veins, when the user sends an authorization application, the user will place the hand in the authorization sensing area set on the gate to perform the corresponding operation. Therefore, when determining whether the target pedestrian is an authorized pedestrian, it is possible to determine whether the target pedestrian's hand is in the authorization sensing area based on the position information of the target pedestrian's hand. When it is determined that the target pedestrian's hand is in the authorization sensing area based on the position information of the target pedestrian's hand, it means that the target pedestrian has performed the authorization application operation. Furthermore, if it is determined that the authorization application received at the current moment is passed, it can also be determined that the target pedestrian is an authorized pedestrian.

[0038] The method for determining authorized pedestrians provided in an embodiment of the present invention collects multiple frames of images in the gate area, tracks pedestrians based on the multiple frames of images, and obtains at least one pedestrian tracking frame. For each pedestrian tracking frame, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is in the authorization sensing area set on the gate, and the authorization application received at the current moment is passed, the target pedestrian is determined to be an authorized pedestrian. In the scenario of obtaining authorization by swiping a card, swiping a QR code, or through palm vein, the pedestrian will place his hand in the authorization sensing area to perform the corresponding operation to send an authorization application. When it is determined that the hand of the target pedestrian is in the authorization sensing area and the authorization application received at the current moment is passed, it means that the target pedestrian has executed the authorization application, thereby accurately determining that the target pedestrian is an authorized pedestrian.

[0039] In addition, by identifying whether the pedestrians passing through are authorized pedestrians, it is possible to effectively prevent card skimming or fare evasion.

[0040] Exemplarily, based on the above embodiment, when it is determined that the target pedestrian is an authorized pedestrian, the gate can be controlled to open to release the target pedestrian.

[0041] Exemplarily, on the basis of the above embodiment, the hand position includes position information of hand feature points, and the hand feature points are used to characterize the hand. When determining that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorized sensing area set on the gate, it can be done in the following way: obtain the position information of the hand feature points, and determine that the hand of the target pedestrian is located in the authorized sensing area when the horizontal coordinate in the position information of the hand feature points is greater than the first horizontal coordinate and less than the second horizontal coordinate, and the vertical coordinate in the position information of the hand feature points is greater than the first vertical coordinate and less than the second vertical coordinate; the first horizontal coordinate is the starting horizontal coordinate of the authorized sensing area, the second horizontal coordinate is the ending horizontal coordinate of the authorized sensing area, the first vertical coordinate is the starting vertical coordinate of the authorized sensing area, and the second vertical coordinate is the ending vertical coordinate of the authorized sensing area.

[0042] Specifically, the hand of the target pedestrian can be characterized by hand feature points, and thus the position information of the hand of the target pedestrian can be characterized by obtaining the position information of the hand feature points. In a possible implementation, when obtaining the position information of the hand feature points, each frame image can be sequentially input into a pedestrian detection model to obtain the detection frame of the target pedestrian and the position information of the hand feature points of the target pedestrian in the detection frame output by the pedestrian detection model; the pedestrian detection model is obtained by training the initial pedestrian detection model based on the sample image, the sample main pedestrian detection frame label in the sample image, and the position label of the hand feature points of the sample main pedestrian; the sample main pedestrian includes pedestrians who can walk independently.

[0043] Specifically, the pedestrian detection model can be, for example, a YOLOv8n-pose model, or a modified model that adds feature point prediction based on yolov5. When training the pedestrian detection model, multiple frames of sample images can be obtained, and the sample main pedestrian detection frame label and the position label of the sample main pedestrian's hand feature points are marked in each sample image. Each sample image is input into the initial pedestrian detection model to obtain the predicted detection frame of the sample main pedestrian and the predicted position of the sample main pedestrian's hand feature points output by the initial pedestrian detection model, and based on the detection frame label, position label, predicted detection frame and predicted position, the loss information is determined, and then based on the loss information, the model parameters of the initial pedestrian detection model are adjusted to obtain the pedestrian detection model.

[0044] Specifically, in the process of data annotation, pedestrians in the sample image obtained from the top-down perspective, including pedestrians holding children, carrying children on their backs, and carrying backpacks and luggage, can be framed and annotated as a whole, and the head height feature points and hand feature points of the sample main pedestrian are correspondingly annotated in a pedestrian annotation frame, that is, the annotated label information includes the detection frame label of the sample main pedestrian in the sample image, the position label of the hand feature points of the sample main pedestrian, and the position label of the head feature points, wherein the hand feature points include left-hand feature points and right-hand feature points. The sample image is input into the initial pedestrian detection model to obtain the predicted detection frame of the sample main pedestrian, the predicted position of the head feature points of the sample main pedestrian, and the predicted position of the hand feature points output by the initial pedestrian detection model. After determining the loss information based on the above-mentioned label information and prediction information, the model parameters of the initial pedestrian detection model are adjusted based on the loss information. By repeatedly executing the above process until the model converges or the number of repetitions reaches a preset number, the final model is determined as the pedestrian detection model.

[0045] Input each frame of real-time collected image into the trained pedestrian detection model in sequence, and the detection frame of the target pedestrian in the pedestrian detection frame, the location information of the target pedestrian's head feature points, and the location information of the target pedestrian's hand feature points will be output. Figure 2 In the above example, we can get the position information of the top feature point A as , the position information of the right hand feature point is , the position information of the left hand feature point is . When training the pedestrian detection model, the pedestrians holding children, carrying children on their backs, and carrying backpacks are selected and annotated as a whole, and the location information of the main pedestrian's head feature points and the location information of the main pedestrian's hand feature points are annotated. Therefore, the target pedestrian selected in the pedestrian detection box output by the pedestrian detection model is the main pedestrian in the image. Among them, the location information of the head feature points is used to determine the height of the target pedestrian.

[0046] It should be noted that there is a correspondence between the detection frame output by the pedestrian detection frame and the main pedestrian's top feature points, left hand feature points, and right hand feature points. When pedestrian tracking is performed based on the detection frame and the pedestrian tracking frame is determined, there will also be a correspondence between the detection frame and the pedestrian tracking frame. Therefore, based on the detection frame, the pedestrian tracking frame will be matched with the pedestrian's top feature points, left hand feature points, and right hand feature points. After the pedestrian tracking frame is matched with the top feature points, left hand feature points, and right hand feature points, when a pedestrian is tracked, the height of the pedestrian can be determined based on the top feature points that match the pedestrian tracking frame of the pedestrian, and whether the pedestrian's hand is located in the authorized sensing area can be determined based on the left hand feature points and right hand feature points that match the pedestrian tracking frame of the pedestrian.

[0047] In this embodiment, when training the pedestrian detection model, the sample main pedestrian detection frame label, the position label of the sample main pedestrian's head feature point and the position label of the sample main pedestrian's hand feature point in the sample image will be marked. In this way, the trained pedestrian detection model can accurately detect the target pedestrian as the main pedestrian in the image, as well as the position of the target pedestrian's head feature points and hand feature points, thereby improving the accuracy of the detection results.

[0048] Continue to refer to Figure 2 As shown, the authorized sensing area can be marked in the image. Assume that the position information of the right hand feature point is , the position information of the left hand feature point is , when the position information of the right-hand feature point is determined to meet and , or the position information of the left hand feature point satisfies and When the target pedestrian's hand is in the authorized sensing area, it can be determined that is the first horizontal coordinate, indicating the starting horizontal coordinate of the authorization sensing area, is the second horizontal axis, indicating the end horizontal axis of the authorized sensing area, is the first ordinate, indicating the starting ordinate of the authorization sensing area, is the second ordinate, indicating the end ordinate of the authorized sensing area.

[0049] In this embodiment, the horizontal coordinate of the hand feature point can be compared with the starting horizontal coordinate and the ending horizontal coordinate of the authorized sensing area, and the vertical coordinate of the hand feature point can be compared with the starting vertical coordinate and the ending vertical coordinate of the authorized sensing area, so as to quickly and accurately determine whether the hand of the target pedestrian is within the authorized sensing area.

[0050] Illustratively, based on the above embodiments, the target pedestrian can also be determined as an unauthorized pedestrian if the following detection conditions are met; wherein the detection conditions include: determining that the target pedestrian's hand is located in the authorization sensing area and no authorization application has been received at the current moment; or, determining that the target pedestrian's hand is located in the authorization sensing area and the authorization application received at the current moment has not been approved, or, determining that the target pedestrian's hand is not located in the authorization sensing area.

[0051] Specifically, in order to prevent pedestrians from making false authorization applications, such as making false card or QR code swiping actions, in one scenario, when the target pedestrian's hand is determined to be located in the authorization sensing area based on the position information of the target pedestrian's hand, but no authorization application is received at the current moment, it means that the target pedestrian has not sent an authorization application, and the target pedestrian is determined to be an unauthorized pedestrian. In another scenario, when the target pedestrian's hand is determined to be located in the authorization sensing area based on the position information of the target pedestrian's hand, and an authorization application is received at the current moment, but the authorization application is not approved, it means that the target pedestrian is an unauthorized person. In yet another scenario, when it is determined that the target pedestrian's hand is not located in the authorization sensing area, it can be determined that the target pedestrian has not sent an authorization application, indicating that the target pedestrian is an unauthorized person. In addition, when the target pedestrian is determined to be an unauthorized person, the gate will not be opened, thereby achieving the purpose of intercepting the target pedestrian and reducing the risk of fare evasion.

[0052] In this embodiment, when it is determined that the target pedestrian has not sent an authorization application or the sent authorization application has not been approved, the target pedestrian can be determined to be an unauthorized person, so that the authorization status of the target pedestrian can be accurately identified.

[0053] Exemplarily, on the basis of the above embodiments, in order to improve the degree of automation and intelligence of gate control, when the target pedestrian is preliminarily determined to be an unauthorized pedestrian based on the hand position, the height of the target pedestrian can also be obtained. When the height of the target pedestrian is less than or equal to the preset height, the authorization status of the target pedestrian is updated to an authorized pedestrian.

[0054] Specifically, in actual scenarios, especially in rail transit scenarios, children can enter the station free of charge. Therefore, after determining that the target pedestrian is an unauthorized pedestrian, the height of the target pedestrian can be further obtained. The height of the target pedestrian can be determined based on the height feature points that match the pedestrian tracking frame of the target pedestrian, or can be determined by any other method that can determine the height of the target pedestrian. The embodiment of the present invention does not limit the method for determining the height of the target pedestrian.

[0055] When the height of the target pedestrian is less than or equal to the preset height, it means that the target pedestrian is a child, and the authorization status of the target pedestrian is updated to an authorized pedestrian. When the height of the target pedestrian is greater than the preset height, it means that the target pedestrian is an adult, and it is determined that the target pedestrian is indeed an unauthorized pedestrian. When the target pedestrian is detected to have the behavior of breaking through the gate, an alarm message will be output. Among them, the preset height can be, for example, 1.2 meters.

[0056] In this embodiment, the authorization status of the target pedestrian may be further determined in combination with the height of the target pedestrian, thereby improving the accuracy of the authorization status of the target pedestrian.

[0057] The authorized pedestrian determination device provided by the present invention is described below. The authorized pedestrian determination device described below and the authorized pedestrian determination method described above can be referenced to each other.

[0058] Figure 3 A schematic diagram of the structure of the authorized pedestrian determination device provided in an embodiment of the present invention, referring to Figure 3 As shown, the authorized pedestrian determination device 300 includes: The acquisition module 11 is used to acquire multiple frames of images in the gate area; A tracking module 12, configured to perform pedestrian tracking based on the multiple frames of images to obtain at least one pedestrian tracking frame; The determination module 13 is used to determine, for each of the pedestrian tracking frames, that the target pedestrian corresponding to the pedestrian tracking frame is an authorized pedestrian when it is determined that the hand of the target pedestrian is located in the authorization sensing area set on the gate and the authorization application received at the current moment is passed.

[0059] In an exemplary embodiment, the hand position includes the position of a hand feature point, and the hand feature point is used to characterize the hand; The determining module 13 is specifically used for: Acquire the position information of the hand feature points; When the horizontal coordinate in the position information of the hand feature point is greater than the first horizontal coordinate and less than the second horizontal coordinate, and the vertical coordinate in the position information of the hand feature point is greater than the first vertical coordinate and less than the second vertical coordinate, it is determined that the hand of the target pedestrian is located in the authorization sensing area; the first horizontal coordinate is the starting horizontal coordinate of the authorization sensing area, the second horizontal coordinate is the ending horizontal coordinate of the authorization sensing area, the first vertical coordinate is the starting vertical coordinate of the authorization sensing area, and the second vertical coordinate is the ending vertical coordinate of the authorization sensing area.

[0060] In an exemplary embodiment, the determination module 13 is specifically configured to: When the following detection conditions are met, determining that the target pedestrian is an unauthorized pedestrian; The detection conditions include: After determining that the hand of the target pedestrian is located in the authorization sensing area and no authorization application is received at the current moment; or,

[0061] After determining that the hand of the target pedestrian is located within the authorization sensing area, and determining that the authorization application received at the current moment is not approved; or,

[0062] It is determined that the hand of the target pedestrian is not located in the authorized sensing area.

[0063] In an exemplary embodiment, the acquisition module is specifically configured to: Each of the images is input into the pedestrian detection model in turn to obtain the detection frame of the target pedestrian output by the pedestrian detection model and the position information of the hand feature points of the target pedestrian in the detection frame; the pedestrian detection model is obtained by training the initial pedestrian detection model based on the sample image, the sample main pedestrian detection frame label in the sample image and the position label of the hand feature points of the sample main pedestrian; the sample main pedestrian includes pedestrians who can walk independently.

[0064] In an exemplary embodiment, the pedestrian detection model is trained based on the following method: Acquire multiple frames of the sample images; Marking the sample main pedestrian detection frame label and the position label of the hand feature point of the sample main pedestrian in each of the sample images; Input each of the sample images into the initial pedestrian detection model to obtain a predicted detection frame of the sample main pedestrian and a predicted position of a hand feature point of the sample main pedestrian output by the initial pedestrian detection model; Determining loss information based on the detection box label, the position label, the predicted detection box, and the predicted position; Based on the loss information, the model parameters of the initial pedestrian detection model are adjusted to obtain the pedestrian detection model.

[0065] In an exemplary embodiment, the apparatus further comprises: an updating module, wherein: An acquisition module, used to acquire the height of the target pedestrian; The updating module is used to update the authorization status of the target pedestrian to an authorized pedestrian when the height of the target pedestrian is less than or equal to a preset height.

[0066] The device of this embodiment can be used to execute the method of any embodiment in the embodiment of the authorized pedestrian determination method side. Its specific implementation process and technical effects are similar to those in the embodiment of the authorized pedestrian determination method side. For details, please refer to the detailed description in the embodiment of the authorized pedestrian determination method side, which will not be repeated here.

[0067] Figure 4 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the authorized pedestrian determination method, which includes: collecting multiple frames of images in the gate area; tracking pedestrians based on the multiple frames of images to obtain at least one pedestrian tracking frame; for each of the pedestrian tracking frames, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate, and the authorization application received at the current moment is passed, determining that the target pedestrian is an authorized pedestrian.

[0068] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0069] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the authorized pedestrian determination method provided by the above-mentioned methods, which includes: collecting multiple frames of images within the gate area; tracking pedestrians based on the multiple frames of images to obtain at least one pedestrian tracking frame; for each of the pedestrian tracking frames, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate, and the authorization application received at the current moment is passed, determining that the target pedestrian is an authorized pedestrian.

[0070] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the authorized pedestrian determination method provided by the above-mentioned methods, the method comprising: acquiring multiple frames of images within the gate area; tracking pedestrians based on the multiple frames of images to obtain at least one pedestrian tracking frame; for each of the pedestrian tracking frames, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate, and the authorization application received at the current moment is passed, determining that the target pedestrian is an authorized pedestrian.

[0071] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0072] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining an authorized pedestrian, characterized in that: include: Collect multiple frames of images in the gate area; A gate is arranged in the gate area; Perform pedestrian tracking based on the multiple frames of images to obtain at least one pedestrian tracking frame; For each of the pedestrian tracking frames, when it is determined that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate and the authorization application received at the current moment is passed, the target pedestrian is determined to be an authorized pedestrian.

2. The method for determining an authorized pedestrian according to claim 1, characterized in that: The hand position includes the position of hand feature points, and the hand feature points are used to characterize the hand; The step of determining that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate includes: Acquire the position information of the hand feature points; When the horizontal coordinate in the position information of the hand feature point is greater than the first horizontal coordinate and less than the second horizontal coordinate, and the vertical coordinate in the position information of the hand feature point is greater than the first vertical coordinate and less than the second vertical coordinate, it is determined that the hand of the target pedestrian is located in the authorization sensing area; the first horizontal coordinate is the starting horizontal coordinate of the authorization sensing area, the second horizontal coordinate is the ending horizontal coordinate of the authorization sensing area, the first vertical coordinate is the starting vertical coordinate of the authorization sensing area, and the second vertical coordinate is the ending vertical coordinate of the authorization sensing area.

3. The authorized pedestrian determination method according to claim 1, characterized in that: The method further comprises: When the following detection conditions are met, determining that the target pedestrian is an unauthorized pedestrian; The detection conditions include: After determining that the hand of the target pedestrian is located in the authorization sensing area and no authorization application is received at the current moment; or, After determining that the hand of the target pedestrian is located within the authorization sensing area, and the authorization application currently received is not approved; or, It is determined that the hand of the target pedestrian is not located in the authorized sensing area.

4. The authorized pedestrian determination method according to claim 2, characterized in that: The obtaining the position information of the hand feature points includes: Each of the images is input into the pedestrian detection model in turn to obtain the detection frame of the target pedestrian output by the pedestrian detection model and the position information of the hand feature points of the target pedestrian in the detection frame; the pedestrian detection model is obtained by training the initial pedestrian detection model based on the sample image, the sample main pedestrian detection frame label in the sample image and the position label of the hand feature points of the sample main pedestrian; the sample main pedestrian includes pedestrians who can walk independently.

5. The authorized pedestrian determination method according to claim 4, characterized in that: The pedestrian detection model is trained based on the following method: Acquire multiple frames of the sample images; Marking the sample main pedestrian detection frame label and the position label of the hand feature point of the sample main pedestrian in each of the sample images; Input each of the sample images into the initial pedestrian detection model to obtain a predicted detection frame of the sample main pedestrian and a predicted position of a hand feature point of the sample main pedestrian output by the initial pedestrian detection model; Determining loss information based on the detection box label, the position label, the predicted detection box, and the predicted position; Based on the loss information, the model parameters of the initial pedestrian detection model are adjusted to obtain the pedestrian detection model.

6. The authorized pedestrian determination method according to claim 3, characterized in that: The method further comprises: Obtaining the height of the target pedestrian; When the height of the target pedestrian is less than or equal to the preset height, the authorization status of the target pedestrian is updated to an authorized pedestrian.

7. The method for determining an authorized pedestrian according to any one of claims 1 to 6, characterized in that: The authorization sensing area includes a palm vein sensing area.

8. An authorized pedestrian determination device, characterized in that: include: An acquisition module, used to acquire multiple frames of images in the gate area; A tracking module, configured to perform pedestrian tracking based on the multiple frames of images to obtain at least one pedestrian tracking frame; The control module is used to determine, for each of the pedestrian tracking frames, that the hand of the target pedestrian corresponding to the pedestrian tracking frame is located in the authorization sensing area set on the gate and the authorization application received at the current moment is passed, and determine that the target pedestrian is an authorized pedestrian.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the authorized pedestrian determination method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the authorized pedestrian determination method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the authorized pedestrian determination method according to any one of claims 1 to 7 is implemented.

Images