Detection Method, Device, Equipment and Medium for Pedestrians Climbing over Traffic Guardrails

Abstract

The present invention discloses a detection method, device, equipment and medium for pedestrians climbing over traffic guardrails, which obtains frame images in a video to be recognized, performs detection on the frame images, and determines the traffic guardrail area in the frame images and the target areas where the two legs of the pedestrians in the frame images are respectively located; if the two target areas of the pedestrians in the frame images are on one side or the other side of the traffic guardrail area, the frame image is determined as a first target image or a second target image, and if the two target areas of the pedestrians in the frame images are respectively on both sides of the traffic guardrail area, the frame image is determined as a third target image; if it is determined that the video to be recognized includes the first target image, the second target image and the third target image, it is determined that the pedestrians climb over the traffic guardrails, thereby realizing real-time acquisition of images of pedestrians climbing over traffic guardrails and improving the accuracy and real-time performance of detecting pedestrians climbing over traffic guardrails.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a detection method, device, equipment and medium for pedestrians climbing over traffic guardrails. Background Art

[0002] With the increase in the number of private cars in society, the incidence of traffic accidents has been rising year by year. A considerable part of them are traffic accidents caused by some citizens who find it troublesome to take a detour, seek convenience, take a shortcut, and randomly climb over guardrails.

[0003] Pedestrians randomly climbing over traffic guardrails may conflict with motor vehicles in the lane and cause accidents, resulting in tragedies; at the same time, it will also cause drivers to brake urgently, leading to vehicle collision and scratching accidents, triggering traffic chaos and congestion. Therefore, how to accurately detect the behavior of pedestrians climbing over traffic guardrails is of great significance.

[0004] In the prior art, the detection of pedestrians climbing over traffic guardrails mainly relies on manual supervision and camera remote monitoring. However, due to the limited scope of manual supervision and the consumption of a large amount of human resources, camera remote monitoring is often used in the prior art.

[0005] When using camera remote monitoring, it mainly adopts an early warning method, system, device and storage medium for pedestrians' illegal climbing over guardrails based on a target tracking algorithm. It mainly detects the video to identify pedestrians and traffic guardrails in the frame images of the video, uses the tracking algorithm to record the trajectories of pedestrians, and determines whether there are overlapping points between the trajectories of pedestrians and traffic guardrails through manual judgment, so as to determine whether pedestrians cross the guardrail.

[0006] However, since the prior art can only determine the result of whether pedestrians cross traffic guardrails, and cannot collect the images of pedestrians crossing traffic guardrails in real time, the accuracy and real-time performance of detecting pedestrians climbing over traffic guardrails are poor. Summary of the Invention

[0007] The present invention provides a detection method, device, equipment and medium for pedestrians climbing over traffic guardrails, so as to solve the problem of poor accuracy and real-time performance in detecting pedestrians climbing over traffic guardrails in the prior art.

[0008] The present invention provides a detection method for pedestrians climbing over traffic guardrails, and the method includes:

[0009] Obtain frame images in a video to be recognized, detect the frame images, and determine the traffic guardrail area in the frame images and the target areas where the two legs of the pedestrians in the frame images are respectively located;

[0010] If two target regions of a pedestrian in the frame image are on one side of the traffic guardrail region, determine the frame image as a first target image; if two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region, determine the frame image as a second target image; if two target regions of the pedestrian in the frame image are on both sides of the traffic guardrail region respectively, determine the frame image as a third target image;

[0011] If it is determined that the to-be-recognized video includes the first target image, the second target image, and the third target image, determine that the pedestrian climbs over the traffic guardrail.

[0012] Further, the detecting the frame image to determine the traffic guardrail region in the frame image and the target regions where the two legs of the pedestrian in the frame image are located respectively includes:

[0013] Based on a preset detection algorithm, determine the pedestrian region and the traffic guardrail region in the frame image;

[0014] Based on a pre-trained neural network model, input the frame image into the neural network model, and determine the target regions where the two legs are located respectively in the pedestrian region in the frame image output by the neural network model.

[0015] Further, before the step of determining the frame image as a first target image if two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region, the method further includes:

[0016] According to the coordinate values of the first target points of each target region among the two target regions of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail region, determine the coordinate values of each second target point on the center line of the traffic guardrail region whose ordinate value is the same as that of each first target point;

[0017] For each first target point, according to the coordinate value of the first target point and the coordinate value of the corresponding second target point, determine the first distance between the first target point and the corresponding second target point;

[0018] According to the first distances between the two first target points and the corresponding second target points respectively, determine the average distance of the two first distances;

[0019] Determining that two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region includes:

[0020] If the abscissa values of the two first target points are both smaller than the abscissa values of the corresponding second target points and the average distance is smaller than a first preset distance threshold, determine that two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region;

[0021] Determining that two target regions of a pedestrian in the frame image are on the other side of the traffic guardrail region includes:

[0022] If the abscissa values of the two first target points are both greater than the abscissa values of the corresponding second target points, and the average distance is greater than a second preset distance threshold, it is determined that the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region;

[0023] Determining that two target regions of a pedestrian in the frame image are respectively on both sides of the traffic guardrail region includes:

[0024] If the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point, and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region.

[0025] Further, after determining that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region and before determining that the frame image is a third target image, the method further includes:

[0026] Determining the minimum area between the first area of the traffic guardrail region in the frame image and the second area of the target region overlapping with the traffic guardrail region;

[0027] Determining the ratio of the third area of the overlapping region in the target region overlapping with the traffic guardrail region to the minimum area as the overlap degree according to the minimum area and the third area of the overlapping region;

[0028] If the overlap degree is greater than a preset overlap degree threshold, perform the subsequent step of determining that the frame image is a third target image.

[0029] Further, the method further includes:

[0030] If the first target image and the second target image are determined, output a voice dissuasion message to dissuade the pedestrian.

[0031] Further, the method further includes:

[0032] Sending the first target image, the second target image, and the third target image to a background server, and performing face recognition by the background server to determine the identity information of the pedestrian.

[0033] Further, the training process of the neural network model includes the following steps:

[0034] For any sample image in the sample set, obtain the sample image and the first label information corresponding to the sample image, where the first label information identifies the target regions where the two legs of the pedestrian are located respectively;

[0035] Input the sample image into the original neural network model to obtain the second label information of the output sample image;

[0036] According to the first label information and the second label information, adjust the parameter values of each parameter of the original neural network model to obtain the trained neural network model.

[0037] Correspondingly, the present invention provides a detection device for a pedestrian climbing over a traffic guardrail. The device includes:

[0038] A detection module for obtaining a frame image in the video to be recognized, detecting the frame image, and determining the traffic guardrail region in the frame image and the target regions where the two legs of the pedestrian in the frame image are located respectively;

[0039] A determination module for determining that the frame image is a first target image if the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region, determining that the frame image is a second target image if the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region, and determining that the frame image is a third target image if the two target regions of the pedestrian in the frame image are on both sides of the traffic guardrail region respectively; if it is determined that the video to be recognized includes the first target image, the second target image, and the third target image, it is determined that the pedestrian climbs over the traffic guardrail.

[0040] Further, the detection module is specifically configured to determine the pedestrian region and the traffic guardrail region in the frame image based on a preset detection algorithm; based on a pre-trained neural network model, input the frame image into the neural network model to determine the target regions where the two legs are located respectively in the pedestrian region of the frame image output by the neural network model.

[0041] Further, before determining that the frame image is a first target image when two target regions of a pedestrian in the frame image are on one side of the traffic guardrail region, the determining module is further configured to determine, according to the coordinate values of the first target points of each target region among the two target regions of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail region, the coordinate values of each second target point on the center line of the traffic guardrail region that has the same ordinate value as each first target point; for each first target point, determine the first distance between the first target point and the corresponding second target point according to the coordinate value of the first target point and the coordinate value of the corresponding second target point; determine the average distance of the two first distances according to the first distances between the two first target points and their corresponding second target points; specifically, if the abscissa values of the two first target points are both less than the abscissa values of the corresponding second target points and the average distance is less than a first preset distance threshold, it is determined that the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region; if the abscissa values of the two first target points are both greater than the abscissa values of the corresponding second target points and the average distance is greater than a second preset distance threshold, it is determined that the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region; if the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region.

[0042] Further, after determining that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region and before determining that the frame image is a third target image, the determining module is further configured to determine the minimum area between the first area of the traffic guardrail region in the frame image and the second area of the target region that overlaps with the traffic guardrail region; determine the ratio of the third area to the minimum area as the overlap degree according to the minimum area and the third area of the overlapping region in the target region that overlaps with the traffic guardrail region; if the overlap degree is greater than a preset overlap degree threshold, perform the subsequent step of determining that the frame image is a third target image.

[0043] Further, the apparatus further includes:

[0044] An output module, configured to output a voice dissuasion message to dissuade the pedestrian if the first target image and the second target image are determined.

[0045] Further, the apparatus further includes:

[0046] A sending module, configured to send the first target image, the second target image, and the third target image to a background server, and the background server performs face recognition on the images and determines the identity information of the pedestrian.

[0047] Further, the apparatus further includes:

[0048] A training module, configured to, for any sample image in a sample set, obtain the sample image and first label information corresponding to the sample image, where the first label information identifies target regions where the two legs of the pedestrian are located; input the sample image into an original neural network model, and obtain second label information of the sample image output by the model; and adjust parameter values of each parameter of the original neural network model according to the first label information and the second label information, so as to obtain the trained neural network model.

[0049] Correspondingly, the present invention provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory complete communication with each other through the communication bus.

[0050] A computer program is stored in the memory. When the program is executed by the processor, the processor implements the steps of any one of the above methods for detecting a pedestrian climbing over a traffic guardrail.

[0051] Correspondingly, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any one of the above methods for detecting a pedestrian climbing over a traffic guardrail are implemented.

[0052] The present invention provides a method, apparatus, device, and medium for detecting a pedestrian climbing over a traffic guardrail. Frame images in a video to be recognized are obtained, the frame images are detected, and a traffic guardrail region in the frame images and target regions where the two legs of the pedestrian in the frame images are located are determined. If the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region, the frame image is determined as a first target image. If the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region, the frame image is determined as a second target image. If the two target regions of the pedestrian in the frame image are on both sides of the traffic guardrail region respectively, the frame image is determined as a third target image. If it is determined that the video to be recognized includes the first target image, the second target image, and the third target image, it is determined that the pedestrian climbs over the traffic guardrail. Since the first target image before the pedestrian climbs over the traffic guardrail, the second target image when the pedestrian climbs over the traffic guardrail, and the third target image after the pedestrian climbs over the traffic guardrail are determined in the video to be recognized in the present invention, images of the pedestrian climbing over the traffic guardrail can be collected in real time, and the accuracy and real-time performance of detecting a pedestrian climbing over the traffic guardrail are improved. Description of the Drawings

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the attached drawings required for description in the embodiments. Obviously, the attached drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other attached drawings can be obtained based on these attached drawings.

[0054] Figure 1 Schematic diagram of the process of a method for detecting pedestrians climbing over traffic guardrails provided by an embodiment of the present invention;

[0055] Figure 2 Schematic diagram of the lengths of each side and the lengths of the diagonals in a quadrilateral provided by an embodiment of the present invention;

[0056] Figure 3 Schematic diagram of the third target image of pedestrians climbing over traffic guardrails provided by an embodiment of the present invention;

[0057] Figure 4 Schematic diagram of the structure of a device for detecting pedestrians climbing over traffic guardrails provided by an embodiment of the present invention;

[0058] Figure 5 Schematic diagram of the framework of a system for detecting pedestrians climbing over traffic guardrails provided by an embodiment of the present invention;

[0059] Figure 6 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0060] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the attached drawings. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0061] In order to improve the accuracy and real-time performance of detecting pedestrians climbing over traffic guardrails, embodiments of the present invention provide a method, device, equipment and medium for detecting pedestrians climbing over traffic guardrails.

[0062] Embodiment 1:

[0063] Figure 1 Schematic diagram of the process of a method for detecting pedestrians climbing over traffic guardrails provided by an embodiment of the present invention, and the process includes the following steps:

[0064] S101: Obtain the frame image in the video to be recognized, detect the frame image, and determine the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively.

[0065] To improve the accuracy and real-time performance of detecting pedestrians climbing over traffic guardrails, a method for detecting pedestrians climbing over traffic guardrails provided by an embodiment of the present invention is applied to an electronic device. The electronic device can be an intelligent terminal device such as a host, a tablet computer, or a smart phone, an image acquisition device such as an intelligent traffic camera, a camera, or a video recorder, or a server. The server can be a local server or a cloud server. The embodiment of the present invention does not limit this.

[0066] The electronic device obtains the video to be recognized. When the electronic device is an image acquisition device, the electronic device itself acquires the video of the traffic guardrail on the road. When the electronic device is an intelligent terminal device or a server, the electronic device obtains the video to be recognized sent by other electronic devices. The other electronic devices can be image acquisition devices connected to the electronic device, or intelligent terminal devices or servers connected to the electronic device. The embodiment of the present invention does not limit this.

[0067] The electronic device obtains the frame image in the video to be recognized and performs detection on the frame image. Specifically, existing detection algorithms can be used to determine the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively. Or a deep learning model can be pre-trained. Based on the pre-trained deep learning model, the frame image is input and the output frame image is obtained. The output frame image is marked with the traffic guardrail area and the target areas where the two legs of the pedestrian are located respectively.

[0068] S102: If the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, determine that the frame image is a first target image. If the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area, determine that the frame image is a second target image. If the two target areas of the pedestrian in the frame image are on both sides of the traffic guardrail area respectively, determine that the frame image is a third target image.

[0069] The electronic device determines the positions of the target regions where the two legs of the pedestrian are located in the frame image relative to the traffic guardrail region in the frame image, that is, whether the target region of the frame image is on the left (left and right in the frame image) side or the right (left and right in the frame image) side of the traffic guardrail region. If both target regions are on the left or right side of the traffic guardrail region, the frame image is determined to be a first target image. If both target regions are on the right or left side of the traffic guardrail region, the frame image is determined to be a second target image. If one of the two target regions is on the left or right side of the traffic guardrail region and the other target region is on the right or left side of the traffic guardrail region, the frame image is determined to be a third target image.

[0070] S103: If it is determined that the to-be-recognized video includes the first target image, the second target image, and the third target image, it is determined that the pedestrian climbs over the traffic guardrail.

[0071] If the electronic device determines that the to-be-recognized video includes the first target image, the second target image, and the third target image, it determines the first target image before the pedestrian climbs over the traffic guardrail, the second target image after climbing over the traffic guardrail, and the third target image when climbing over the traffic guardrail. Therefore, it can directly determine that the pedestrian climbs over the traffic guardrail.

[0072] In the embodiment of the present invention, a frame image in the to-be-recognized video is obtained, the frame image is detected, and the traffic guardrail region in the frame image and the target regions where the two legs of the pedestrian are located in the frame image are determined. For the frame image, if the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region, the frame image is determined to be a first target image. If the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region, the frame image is determined to be a second target image. If the two target regions of the pedestrian in the frame image are on both sides of the traffic guardrail region, the frame image is determined to be a third target image. If it is determined that the to-be-recognized video includes the first target image, the second target image, and the third target image, it is determined that the pedestrian climbs over the traffic guardrail. Since in the present invention, the first target image before the pedestrian climbs over the traffic guardrail, the second target image when climbing over the traffic guardrail, and the third target image after climbing over the traffic guardrail are determined in the to-be-recognized video, the image of the pedestrian climbing over the traffic guardrail can be collected in real time, and the accuracy and real-time performance of detecting the pedestrian climbing over the traffic guardrail are improved.

[0073] Embodiment 2:

[0074] In order to detect the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian are located respectively, based on the above embodiments, in the embodiments of the present invention, the detection of the frame image to determine the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively includes:

[0075] Based on a preset detection algorithm, determine the pedestrian area and the traffic guardrail area in the frame image;

[0076] Based on a pre-trained neural network model, input the frame image into the neural network model, and determine the target areas where the two legs are located respectively in the pedestrian area of the frame image output by the neural network model.

[0077] In order to detect the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian are located, the electronic device detects the frame image based on a preset detection algorithm to determine the traffic guardrail area and the pedestrian area in the frame image.

[0078] In order to detect the target areas where the two legs are located respectively in the pedestrian area of the frame image, the electronic device stores a pre-trained neural network model, which can be used to determine the target areas where the two legs are located in the pedestrian area, or can be used to determine the areas of each key part in the pedestrian area. The key parts include the head, shoulders, two arms and two legs.

[0079] Among them, the neural network model can be constructed by using a CNN convolutional neural network through the TensorFlow tool. The neural network model includes 4 convolutional layers, 3 pooling layers and a fully connected layer. The first two convolutional layers use a 5*5 convolutional kernel, and the last two convolutional layers use a 3*3 convolutional layer, with a stride of 1 for both. The pooling layer is an average pooling layer, the activation function uses the sigmoid function, and the fully connected neural network of the last layer uses the softmax function. At the same time, the categorical cross-entropy function is used as the loss function.

[0080] The electronic device inputs the frame image into the neural network model based on the pre-stored neural network model. After the convolutional processing of the convolutional layer, the pooling processing of the pooling layer and the fully connected processing of the fully connected layer of the neural network model, it determines the target areas where the two legs are located respectively in the pedestrian area of the output frame image.

[0081] As a possible implementation manner, in the embodiments of the present invention, the electronic device also processes the frame image based on a preset target tracking algorithm to determine the motion trajectory of each pedestrian, as well as the pedestrian area and the target areas of the two legs of each pedestrian at each moment.

[0082] Embodiment 3:

[0083] Based on the above embodiments, in order to determine whether the two target regions of the pedestrian in the frame image are on one side or both sides of the traffic guardrail region, in the embodiments of the present invention, before determining that the frame image is the first target image if the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region, the method further includes:

[0084] According to the coordinate values of the first target points of each target region among the two target regions of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail region, determine the coordinate values of each second target point on the center line of the traffic guardrail region that has the same ordinate value as each first target point;

[0085] For each first target point, according to the coordinate value of the first target point and the coordinate value of the corresponding second target point, determine the first distance between the first target point and the corresponding second target point;

[0086] According to the first distances between the two first target points and the corresponding second target points respectively, determine the average distance of the two first distances;

[0087] Determining that the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region includes:

[0088] If the abscissa values of the two first target points are both less than the abscissa values of the corresponding second target points and the average distance is less than the first preset distance threshold, it is determined that the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region;

[0089] Determining that the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region includes:

[0090] If the abscissa values of the two first target points are both greater than the abscissa values of the corresponding second target points and the average distance is greater than the second preset distance threshold, it is determined that the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region;

[0091] Determining that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region includes:

[0092] If the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region.

[0093] In order to determine the positions of two target areas of pedestrians in the frame image relative to the traffic guardrail area, the electronic device determines the coordinate value of the first target point of each of the two target areas of the pedestrians in the frame image according to the coordinate value of each pixel point in the frame image, wherein the first target point can be the midpoint of the target area or the midpoint of the bottom frame line of the target area; and determines the coordinate value of each point on the center line of the traffic guardrail area, and determines each second target point in each point with the same longitudinal coordinate value as each first target point according to the longitudinal coordinate value of the first target point of each first target area and the coordinate value of each point on the center line of the traffic guardrail area, and determines the coordinate value of each second target point.

[0094] For each first target point, the electronic device determines the first distance between the first target point and the corresponding second target point based on the coordinate value of the first target point and the corresponding second target point; and determines the average distance of the two first distances based on the first distances between the two first target points and the corresponding second target points.

[0095] As a possible implementation, in an embodiment of the present invention, the electronic device can also determine the intersection of the extension line connecting the two first target points and the center line as the second target point based on the coordinate value of the first target point in each of the two target areas of the pedestrian in the frame image and the center line of the traffic guardrail area, and determine the coordinate value of the second target point; determine the average distance of the first distance between each first target point and the second target point based on the coordinate value of each first target point and the coordinate value of the second target point.

[0096] According to the horizontal coordinate values of the two first target points and the horizontal coordinate values of the second target points corresponding to the two first target points, it is determined whether the horizontal coordinate values of the two first target points are both smaller than the horizontal coordinate values of the corresponding second target points. If so, it is determined that the pedestrian in the frame image is close to the traffic guardrail, and it is determined whether the average distance is less than the first preset distance threshold. If so, it is determined that the two target areas of the pedestrian in the frame image are located on one side of the traffic guardrail area, wherein the first preset distance threshold is pre-set by the user. If it is desired to improve the accuracy of determining whether the pedestrian has climbed over the traffic guardrail, the first preset distance threshold can be set to a smaller value. If it is desired to improve the real-time performance of determining whether the pedestrian has climbed over the traffic guardrail, the first preset distance threshold can be set to a larger value.

[0097] Determine whether the abscissa values of two first target points are both greater than the abscissa value of the corresponding second target point. If so, it is determined that the pedestrian in the frame image is moving away from the traffic guardrail, and it is determined whether the average distance is greater than a second preset distance threshold. If so, it is determined that the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region, where the second preset distance threshold is set by the user in advance. If it is desired to improve the real-time performance of determining whether a pedestrian climbs over the traffic guardrail, the second preset distance threshold can be set smaller. If it is desired to improve the accuracy of determining whether a pedestrian climbs over the traffic guardrail, the second preset distance threshold can be set larger. The first preset distance threshold and the second preset distance threshold can be the same or different, and the embodiments of the present invention do not limit this.

[0098] According to the abscissa value of each first target point among the two first target points and the abscissa value of the corresponding second target point, if the abscissa value of any one first target point is less than the abscissa value of the corresponding second target point and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region.

[0099] The following uses a specific embodiment to illustrate the method for detecting a pedestrian climbing over a traffic guardrail of the present invention. The electronic device normalizes each target region of the pedestrian in the frame image and the traffic guardrail region to obtain a straight line as the center line of the traffic guardrail region, and records the center line as L. The bottom midpoints of the two target regions are respectively recorded as pl and pr. Calculate the distances from pl and pr to L, and take the average value D1 as the distance between the pedestrian and the guardrail. If D1 is less than a certain set threshold and pl and pr are on the same side of L, it is determined that the pedestrian is approaching the guardrail and there is a possibility of climbing over the guardrail.

[0100] If pl and pr are on both sides of L and the coincidence degree is greater than a preset coincidence degree threshold, it is determined that the pedestrian is climbing over the traffic guardrail; continue to calculate the average distance D2 from pr and pl to L. When D2 is greater than the second distance threshold and both pr and pl are on the other side of L, it is determined that the pedestrian has the behavior of climbing over the traffic guardrail.

[0101] Embodiment 4:

[0102] In order to more accurately determine the third target image, on the basis of the above embodiments, in the embodiments of the present invention, after the two target regions of the pedestrian in the frame image are respectively on both sides of the traffic guardrail region and before determining that the frame image is the third target image, the method further includes:

[0103] Determine the minimum area between the first area of the traffic guardrail area and the second area of the target area that overlaps with the traffic guardrail area in the frame image;

[0104] Determine the ratio of the third area to the minimum area as the overlap degree according to the minimum area and the third area of the overlapping area in the target area that overlaps with the traffic guardrail area;

[0105] If the overlap degree is greater than the preset overlap degree threshold, perform the subsequent step of determining the frame image as the third target image.

[0106] In order to accurately determine whether a pedestrian is climbing over a traffic guardrail in a frame image, in an embodiment of the present invention, the electronic device determines the length of each side and the length of the two diagonals of each side of the traffic guardrail area according to the traffic guardrail area in the frame image and the target area where the two legs of the pedestrian are located; according to the length of each side and the length of the two diagonals, and the pre-stored area calculation formula, determine the first area of the traffic guardrail area.

[0107] The electronic device determines the target area that overlaps with the traffic guardrail area in the two target areas according to the traffic guardrail area in the frame image and the target area where the two legs of the pedestrian are located, and determines the second area of the target area that overlaps with the traffic guardrail area according to the length of each side and the length of the two diagonals of the target area that overlaps with the traffic guardrail area and the pre-stored area calculation formula, and determines the minimum area between the first area and the second area according to the first area and the second area.

[0108] The electronic device determines the length of each side and the length of the two diagonals of the overlapping area according to the overlapping area in the target area that overlaps with the traffic guardrail area, and determines the third area of the overlapping area according to the length of each side and the length of the two diagonals of the overlapping area and the pre-stored area calculation formula.

[0109] Wherein, since the traffic guardrail area, the target area, and the overlapping area are all quadrilateral areas, the pre-stored area calculation formula is Where m and n represent the lengths of the two diagonals of the quadrilateral, b and d represent the lengths of the two opposite sides of the quadrilateral, and a and c represent the lengths of the other two opposite sides of the quadrilateral.

[0110] Figure 2 For a schematic diagram of the length of each side and the length of the diagonal in a quadrilateral provided by an embodiment of the present invention, as Figure 2As shown, the length of side AB in the quadrilateral is a, the length of side BC is b, the length of side CD is c, the length of side AD is d, the length of diagonal AC is m, and the length of diagonal BD is n.

[0111] According to the third area and the minimum area among the first area and the second area, determine the proportional value of the third area and the minimum area, and determine this proportional value as the coincidence degree between the target area and the traffic guardrail area. The calculation formula for determining the coincidence degree between the target area and the traffic guardrail area is where S c represents the third area of the overlapping area between the target area and the traffic guardrail area, S h represents the first area of the traffic guardrail area, S x represents the second area of the target area of the pedestrian's leg that overlaps with the traffic guardrail area.

[0112] According to the calculated coincidence degree and the preset coincidence degree threshold, the electronic device determines whether the coincidence degree is greater than the preset coincidence degree threshold. If the coincidence degree is greater than the preset coincidence degree threshold, it is determined that the pedestrian in the frame image is climbing over the traffic guardrail, and the frame image is determined as the third target image.

[0113] Figure 3 FIG. is a schematic diagram of a third target image of a pedestrian climbing over a traffic guardrail provided by an embodiment of the present invention. As Figure 3 shown, Figure 3 the longest quadrilateral area in is the traffic guardrail area. The center line of the traffic guardrail area is parallel to the long side. The black area is the overlapping area between the traffic guardrail area and the target area of the pedestrian's leg. The quadrilateral area with a smaller length is the target area of the pedestrian's leg.

[0114] Embodiment 5:

[0115] In order to dissuade pedestrians from climbing over the traffic guardrail, based on the above embodiments, in an embodiment of the present invention, the method further includes:

[0116] If the first target image and the second target image are determined, output a voice dissuasion message to dissuade the pedestrian.

[0117] In order to dissuade pedestrians from climbing over the traffic guardrail, after the electronic device determines the first target image and the second target image, it outputs a voice dissuasion message to dissuade the pedestrians who are climbing over the traffic guardrail. Specifically, it controls the voice playback function of the image acquisition device to play the traffic guardrail area.

[0118] Embodiment 6:

[0119] In order to determine the identity information of a pedestrian who climbs over a traffic guardrail, based on the above embodiments, in an embodiment of the present invention, the first target image, the second target image, and the third target image are sent to a background server, and the background server performs face recognition to determine the identity information of the pedestrian.

[0120] In order to determine the identity information of a pedestrian in the area where the traffic guardrail is climbed over, the electronic device also needs to send the target image to the background server every time a target image is determined, that is, send the first target image, the second target image, and the third target image to the background server and save them. The background server performs face recognition on the first target image, the second target image, and the third target image, and determines the identity information of the pedestrian according to the face recognition result.

[0121] As a possible implementation manner, in an embodiment of the present invention, after the background server determines the identity information of the pedestrian, it may also send the identity information of the pedestrian to the server of the supervision department, and impose a penalty on the pedestrian who climbs over the traffic guardrail and conduct relevant safety education when the pedestrian is found.

[0122] Embodiment 7:

[0123] In order to implement the training of the neural network model, based on the above embodiments, in an embodiment of the present invention, the training process of the neural network model includes the following steps:

[0124] For any sample image in the sample set, obtain the sample image and the first label information corresponding to the sample image, where the first label information identifies the target areas where the two legs of the pedestrian are located;

[0125] Input the sample image into the original neural network model, and obtain the second label information of the output sample image;

[0126] According to the first label information and the second label information, adjust the parameter values of the various parameters of the original neural network model to obtain the trained neural network model.

[0127] In order to implement the training of the neural network model, an embodiment of the present invention stores a sample set for training. The sample images in the sample set include pedestrian area images with position information of the leg areas of pedestrians. The first label information of the sample images in the sample set is manually pre-annotated, where the first label information is used to identify the position information of the leg areas in the sample images.

[0128] In an embodiment of the present invention, after obtaining any sample image in the sample set and the first label information of the sample image, the sample image is input into the original neural network model, and the original neural network model outputs the second label information of the sample image. The second label information identifies the leg area of the pedestrian in the sample image identified by the original neural network model.

[0129] After determining the second label information of the sample image according to the original neural network model, the original neural network model is trained according to the second label information and the first label information of the sample image to adjust the parameter values of various parameters of the original neural network model.

[0130] The above operation is performed on each sample image included in the sample set for training the original neural network model, and when the preset conditions are met, a trained neural network model is obtained. The preset condition may be that the number of sample images in the sample set whose first label information and second label information are consistent after training the sample images with the original neural network model is greater than a set number; or the number of iterations for training the original neural network model reaches the set maximum number of iterations, etc. Specifically, this application does not impose any restrictions on this.

[0131] As a possible implementation method, when training the original neural network model, the sample images in the sample set can be divided into training sample images and test sample images. The original recognition model is first trained based on the training sample images, and then the reliability of the trained recognition model is tested based on the test sample images.

[0132] As another possible implementation, the sample images in the sample set saved in the embodiment of the present invention include pedestrian area images with position information of each key part of the pedestrian, the key parts including the head, shoulders, left and right arms, and left and right legs, and data enhancement is performed on the sample images in the sample set, specifically, data enhancement is performed by means of horizontal flipping and vertical flipping.

[0133] Embodiment 8:

[0134] Figure 4 A schematic diagram of a detection device for detecting a pedestrian climbing over a traffic barrier provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device comprises:

[0135] The detection module 401 is used to obtain a frame image in the video to be identified, detect the frame image, and determine the target area where the traffic guardrail area in the frame image and the two legs of the pedestrian in the frame image are respectively located;

[0136] A determination module 402, configured to determine that the frame image is a first target image if two target regions of a pedestrian in the frame image are on one side of the traffic guardrail region, determine that the frame image is a second target image if the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region, and determine that the frame image is a third target image if the two target regions of the pedestrian in the frame image are on both sides of the traffic guardrail region respectively; if it is determined that the to-be-recognized video includes the first target image, the second target image, and the third target image, determine that the pedestrian climbs over the traffic guardrail.

[0137] Further, the detection module 401 is specifically configured to determine the pedestrian region and the traffic guardrail region in the frame image based on a preset detection algorithm; input the frame image into a pre-trained neural network model based on the pre-trained neural network model, and determine target regions where two legs are respectively located in the pedestrian region of the frame image output by the neural network model.

[0138] Further, before determining that the frame image is a first target image if two target regions of a pedestrian in the frame image are on one side of the traffic guardrail region, the determination module 402 is further configured to determine coordinate values of each second target point on the center line of the traffic guardrail region that has the same ordinate value as each first target point according to the coordinate values of the first target points of each target region of the two target regions of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail region; for each first target point, determine a first distance between the first target point and the corresponding second target point according to the coordinate value of the first target point and the coordinate value of the corresponding second target point; determine an average distance between the two first distances according to the first distances between the two first target points and the corresponding second target points respectively; specifically configured to determine that the two target regions of the pedestrian in the frame image are on one side of the traffic guardrail region if the abscissa values of the two first target points are both smaller than the abscissa values of the corresponding second target points and the average distance is smaller than a first preset distance threshold; determine that the two target regions of the pedestrian in the frame image are on the other side of the traffic guardrail region if the abscissa values of the two first target points are both larger than the abscissa values of the corresponding second target points and the average distance is larger than a second preset distance threshold; determine that the two target regions of the pedestrian in the frame image are on both sides of the traffic guardrail region respectively if the abscissa value of any one of the two first target points is smaller than the abscissa value of the corresponding second target point and the abscissa value of the other first target point is larger than the abscissa value of the corresponding second target point.

[0139] Further, after the determining module 402 determines that two target regions of a pedestrian in the frame image are respectively located on both sides of the traffic guardrail region, before determining that the frame image is a third target image, the determining module 402 is further configured to determine the minimum area between the first area of the traffic guardrail region in the frame image and the second area of the target region that overlaps with the traffic guardrail region; determine the ratio of the third area of the overlapping region in the target region that overlaps with the traffic guardrail region to the minimum area as the overlapping degree according to the minimum area and the third area; if the overlapping degree is greater than a preset overlapping degree threshold, then perform the subsequent step of determining that the frame image is a third target image.

[0140] Further, the apparatus further includes:

[0141] An output module 403, configured to output a voice dissuasion message to dissuade the pedestrian if the first target image and the second target image are determined.

[0142] Further, the apparatus further includes:

[0143] A sending module 404, configured to send the first target image, the second target image, and the third target image to a background server, and the background server performs face recognition and determines the identity information of the pedestrian.

[0144] Further, the apparatus further includes:

[0145] A training module 405, configured to, for any sample image in a sample set, obtain the sample image and first label information corresponding to the sample image, where the first label information identifies target regions where the two legs of the pedestrian are respectively located; input the sample image into an original neural network model, and obtain second label information of the output sample image; adjust the parameter values of each parameter of the original neural network model according to the first label information and the second label information, and obtain the trained neural network model.

[0146] Figure 5 The figure is a schematic framework diagram of a detection system for pedestrians climbing over a traffic guardrail provided by an embodiment of the present invention. As Figure 5 shown, the system includes a front-end intelligent traffic camera 501, a background server 502, and a server 503 of a supervision department; the front-end intelligent traffic camera 501 is configured to collect a video to be recognized, obtain a frame image, determine whether a pedestrian climbs over a traffic guardrail based on a pre-stored detection algorithm and target tracking algorithm, and a pre-trained neural network model, and output a voice dissuasion message when it is found that a pedestrian climbs over a traffic guardrail; send the determined first target image, second target image, and third target image to the background server.

[0147] The background server 502 is used to save the first target image, the second target image, and the third target image, perform face recognition on pedestrians to determine the identity information of the pedestrians, and send the identity information of the pedestrians, the first target image, the second target image, and the third target image to the server of the regulatory department.

[0148] The server 503 of the regulatory department is used to receive the identity information of the pedestrians, the first target image, the second target image, and the third target image, and determine the pedestrians who climb over the traffic guardrail through big data comparison, so as to impose penalties on the pedestrians and conduct relevant safety education.

[0149] Embodiment 9:

[0150] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. On the basis of the above embodiments, the present application further provides an electronic device, as Figure 6 shown, including: a processor 601, a communication interface 602, a memory 603, and a communication bus 604. Among them, the processor 601, the communication interface 602, and the memory 603 complete communication with each other through the communication bus 604.

[0151] A computer program is stored in the memory 603. When the program is executed by the processor 601, the processor 601 is caused to execute the following steps:

[0152] Obtain a frame image in the video to be recognized, detect the frame image, and determine the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively;

[0153] If the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, determine that the frame image is the first target image. If the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area, determine that the frame image is the second target image. If the two target areas of the pedestrian in the frame image are on both sides of the traffic guardrail area respectively, determine that the frame image is the third target image;

[0154] If it is determined that the video to be recognized includes the first target image, the second target image, and the third target image, determine that the pedestrian climbs over the traffic guardrail.

[0155] Further, the processor 601 is specifically used for detecting the frame image and determining the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively, including:

[0156] Based on a preset detection algorithm, determine the pedestrian area and the traffic guardrail area in the frame image;

[0157] Based on a pre-trained neural network model, input the frame image into the neural network model, and determine the target areas where the two legs in the pedestrian area of the frame image output by the neural network model are located respectively.

[0158] Further, before the processor 601 further determines that the frame image is a first target image if the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, the method further includes:

[0159] According to the coordinate values of the first target points of each target area among the two target areas of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail area, determine the coordinate values of each second target point on the center line of the traffic guardrail area that has the same ordinate value as each first target point;

[0160] For each first target point, determine the first distance between the first target point and the corresponding second target point according to the coordinate value of the first target point and the coordinate value of the corresponding second target point;

[0161] According to the first distances between the two first target points and the corresponding second target points respectively, determine the average distance of the two first distances;

[0162] Determining that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area includes:

[0163] If the abscissa values of the two first target points are both less than the abscissa values of the corresponding second target points and the average distance is less than a first preset distance threshold, determine that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area;

[0164] Determining that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area includes:

[0165] If the abscissa values of the two first target points are both greater than the abscissa values of the corresponding second target points and the average distance is greater than a second preset distance threshold, determine that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area;

[0166] Determining that the two target areas of the pedestrian in the frame image are respectively on both sides of the traffic guardrail area includes:

[0167] If the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point, and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region.

[0168] Further, after the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region and before it is determined that the frame image is a third target image, the processor 601 is further configured to:

[0169] Determine the minimum area of the first area of the traffic guardrail region in the frame image and the second area of the target region that overlaps with the traffic guardrail region;

[0170] Determine the ratio of the third area of the overlapping region in the target region that overlaps with the traffic guardrail region to the minimum area as the overlap degree according to the minimum area;

[0171] If the overlap degree is greater than a preset overlap degree threshold, perform the subsequent step of determining that the frame image is a third target image.

[0172] Further, if the first target image and the second target image are determined, the processor 601 is further configured to output a voice dissuasion message to dissuade the pedestrian.

[0173] Further, the processor 601 is further configured to send the first target image, the second target image, and the third target image to a background server for face recognition by the background server and determination of the identity information of the pedestrian.

[0174] Further, the processor 601 is specifically configured that the training process of the neural network model includes the following steps:

[0175] For any sample image in the sample set, obtain the sample image and the first label information corresponding to the sample image, where the first label information identifies the target regions where the two legs of the pedestrian are respectively located;

[0176] Input the sample image into the original neural network model to obtain the second label information of the output sample image;

[0177] Adjust the parameter values of the parameters of the original neural network model according to the first label information and the second label information to obtain the trained neural network model.

[0178] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity in illustration, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus.

[0179] The communication interface 602 is used for communication between the above electronic device and other devices.

[0180] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0181] The above processor may be a general-purpose processor, including a central processing unit, a Network Processor (NP), etc.; it may also be a Digital Signal Processing (DSP), an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0182] Embodiment 10:

[0183] Based on the above embodiments, the present application further provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor is caused to execute the following steps:

[0184] Obtain a frame image in the video to be recognized, detect the frame image, and determine the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively;

[0185] If the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, determine that the frame image is a first target image; if the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area, determine that the frame image is a second target image; if the two target areas of the pedestrian in the frame image are on both sides of the traffic guardrail area respectively, determine that the frame image is a third target image;

[0186] If it is determined that the video to be recognized includes the first target image, the second target image, and the third target image, it is determined that a pedestrian climbs over a traffic guardrail.

[0187] Further, the detecting the frame image to determine the traffic guardrail area in the frame image and the target areas where the two legs of the pedestrian in the frame image are located respectively includes:

[0188] Based on a preset detection algorithm, determine the pedestrian area and the traffic guardrail area in the frame image;

[0189] Based on a pre-trained neural network model, input the frame image into the neural network model, and determine the target areas where the two legs in the pedestrian area of the frame image output by the neural network model are located respectively.

[0190] Further, before determining that the frame image is the first target image if the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, the method further includes:

[0191] According to the coordinate values of the first target points of each target area among the two target areas of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail area, determine the coordinate values of each second target point on the center line of the traffic guardrail area that has the same ordinate value as each first target point;

[0192] For each first target point, according to the coordinate value of the first target point and the coordinate value of the corresponding second target point, determine the first distance between the first target point and the corresponding second target point;

[0193] According to the first distances between the two first target points and the corresponding second target points respectively, determine the average distance of the two first distances;

[0194] Determining that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area includes:

[0195] If the abscissa values of the two first target points are both less than the abscissa values of the corresponding second target points and the average distance is less than the first preset distance threshold, it is determined that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area;

[0196] Determining that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area includes:

[0197] If the abscissa values of the two first target points are both greater than the abscissa values of the corresponding second target points and the average distance is greater than the second preset distance threshold, it is determined that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area;

[0198] Determining that two target regions of a pedestrian in the frame image are respectively located on both sides of the traffic guardrail region includes:

[0199] If the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point, and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region.

[0200] Further, after determining that the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region and before determining that the frame image is the third target image, the method further includes:

[0201] According to the first area of the traffic guardrail region in the frame image and the second area of the target region that overlaps with the traffic guardrail region, determine the minimum area of the first area and the second area;

[0202] According to the minimum area and the third area of the overlapping region in the target region that overlaps with the traffic guardrail region, determine the ratio of the third area to the minimum area as the overlapping degree;

[0203] If the overlapping degree is greater than a preset overlapping degree threshold, perform the subsequent step of determining that the frame image is the third target image.

[0204] Further, the method further includes:

[0205] If the first target image and the second target image are determined, output a voice dissuasion message to dissuade the pedestrian.

[0206] Further, the method further includes:

[0207] Send the first target image, the second target image, and the third target image to the background server, and the background server performs face recognition and determines the identity information of the pedestrian.

[0208] Further, the training process of the neural network model includes the following steps:

[0209] For any sample image in the sample set, obtain the sample image and the first label information corresponding to the sample image, where the first label information identifies the target regions where the two legs of the pedestrian are respectively located;

[0210] Input the sample image into the original neural network model, and obtain the second label information of the output sample image;

[0211] Adjust the parameter values of each parameter of the original neural network model according to the first tag information and the second tag information to obtain the trained neural network model.

[0212] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0213] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more of the processes or Figure 1 multiple processes and / or blocks.

[0214] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one or more of the processes or Figure 1 multiple processes and / or blocks.

[0215] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the processes or Figure 1 multiple processes and / or blocks.

[0216] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A detection method for pedestrians climbing over traffic guardrails, characterized in that, The method includes: Obtaining a frame image in a video to be recognized, detecting the frame image, and determining a traffic guardrail area in the frame image and target areas where two legs of a pedestrian in the frame image are located respectively; If the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, determining the frame image as a first target image; if the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area, determining the frame image as a second target image; if the two target areas of the pedestrian in the frame image are on both sides of the traffic guardrail area respectively, determining the frame image as a third target image; If it is determined that the video to be recognized includes the first target image, the second target image, and the third target image, determining that the pedestrian climbs over the traffic guardrail.

2. The method according to claim 1, wherein The detecting the frame image and determining the traffic guardrail area in the frame image and the target areas where two legs of the pedestrian in the frame image are located respectively includes: Based on a preset detection algorithm, determining a pedestrian area and a traffic guardrail area in the frame image; Based on a pre-trained neural network model, inputting the frame image into the neural network model and determining the target areas where two legs in the pedestrian area of the frame image output by the neural network model are located respectively.

3. The method according to claim 1, characterized in that Before determining that the frame image is a first target image if the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, the method further includes: According to the coordinate values of first target points in each of the two target areas of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail area, determining the coordinate values of second target points on the center line of the traffic guardrail area that have the same ordinate value as each first target point; For each first target point, determining a first distance between the first target point and the corresponding second target point according to the coordinate value of the first target point and the coordinate value of the corresponding second target point; Determining an average distance between the two first distances according to the first distances between the two first target points and their corresponding second target points respectively; Determining that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area includes: If the abscissa values of the two first target points are both less than the abscissa values of the corresponding second target points and the average distance is less than a first preset distance threshold, determining that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area; Determining that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area includes: If the abscissa values of the two first target points are both greater than the abscissa values of the corresponding second target points and the average distance is greater than a second preset distance threshold, determining that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area; Determining that the two target areas of the pedestrian in the frame image are on both sides of the traffic guardrail area respectively includes: If the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point, and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region.

4. The method according to claim 1, characterized in that, After it is determined that the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region, and before it is determined that the frame image is a third target image, the method further includes: Determine the minimum area between the first area of the traffic guardrail region in the frame image and the second area of the target region that overlaps with the traffic guardrail region; According to the minimum area and the third area of the overlapping region in the target region that overlaps with the traffic guardrail region, determine that the ratio of the third area to the minimum area is the overlapping degree; If the overlapping degree is greater than a preset overlapping degree threshold, perform the subsequent step of determining that the frame image is a third target image.

5. The method according to claim 1, characterized in that, The method further includes: If the first target image and the second target image are determined, output a voice dissuasion message to dissuade the pedestrian.

6. The method according to claim 1, wherein The method further includes: Send the first target image, the second target image, and the third target image to the background server, and the background server performs face recognition and determines the identity information of the pedestrian.

7. The method according to claim 2, wherein The training process of the neural network model includes the following steps: For any sample image in the sample set, obtain the sample image and the first label information corresponding to the sample image, where the first label information identifies the target regions where the two legs of the pedestrian are respectively located; Input the sample image into the original neural network model, and obtain the second label information of the output sample image; According to the first label information and the second label information, adjust the parameter values of the various parameters of the original neural network model to obtain the trained neural network model.

8. A detection device for pedestrians climbing over traffic guardrails, characterized in that, The device includes: A detection module, configured to obtain a frame image in a video to be recognized, detect the frame image, and determine the traffic guardrail region in the frame image and the target regions where the two legs of the pedestrian in the frame image are respectively located; A determination module, configured to determine that the frame image is a first target image if the two target regions of the pedestrian in the frame image are located on one side of the traffic guardrail region, determine that the frame image is a second target image if the two target regions of the pedestrian in the frame image are located on the other side of the traffic guardrail region, and determine that the frame image is a third target image if the two target regions of the pedestrian in the frame image are respectively located on both sides of the traffic guardrail region; if it is determined that the video to be recognized includes the first target image, the second target image, and the third target image, it is determined that the pedestrian climbs over the traffic guardrail.

9. The device according to claim 8, characterized in that The detection module is specifically configured to determine the pedestrian area and the traffic guardrail area in the frame image based on a preset detection algorithm; based on a pre-trained neural network model, input the frame image into the neural network model to determine the target areas where the two legs in the pedestrian area of the frame image output by the neural network model are located respectively.

10. The device according to claim 8, characterized in that, Before determining that the frame image is the first target image if the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area, the determination module is further configured to determine the coordinate values of each second target point on the center line of the traffic guardrail area that has the same ordinate value as each first target point according to the coordinate values of each first target point in the two target areas of the pedestrian in the frame image and the coordinate values of each point on the center line of the traffic guardrail area; for each first target point, determine the first distance between the first target point and the corresponding second target point according to the coordinate value of the first target point and the coordinate value of the corresponding second target point; determine the average distance of the two first distances according to the first distances between the two first target points and the corresponding second target points respectively. Specifically, if the abscissa values of the two first target points are both less than the abscissa value of the corresponding second target point and the average distance is less than the first preset distance threshold, it is determined that the two target areas of the pedestrian in the frame image are on one side of the traffic guardrail area; if the abscissa values of the two first target points are both greater than the abscissa value of the corresponding second target point and the average distance is greater than the second preset distance threshold, it is determined that the two target areas of the pedestrian in the frame image are on the other side of the traffic guardrail area; if the abscissa value of any one of the two first target points is less than the abscissa value of the corresponding second target point and the abscissa value of the other first target point is greater than the abscissa value of the corresponding second target point, it is determined that the two target areas of the pedestrian in the frame image are respectively on both sides of the traffic guardrail area.

11. The device according to claim 8, characterized in that, Before determining that the frame image is the third target image after the two target areas of the pedestrian in the frame image are respectively on both sides of the traffic guardrail area, the determination module is further configured to determine the minimum area between the first area of the traffic guardrail area in the frame image and the second area of the target area that overlaps with the traffic guardrail area; determine the ratio of the third area to the minimum area as the overlap degree according to the minimum area and the third area of the overlapping area in the target area that overlaps with the traffic guardrail area; if the overlap degree is greater than the preset overlap degree threshold, perform the subsequent step of determining that the frame image is the third target image.

12. The device according to claim 8, characterized in that, The device further includes: An output module, configured to output a voice dissuasion message to dissuade the pedestrian if the first target image and the second target image are determined.

13. The device according to claim 8, wherein The device further includes: A sending module, configured to send the first target image, the second target image, and the third target image to a background server, and the background server performs face recognition on the images and determines the identity information of the pedestrian.

14. The device according to claim 9, wherein, The device further includes: A training module, configured to, for any sample image in a sample set, obtain the sample image and first label information corresponding to the sample image, where the first label information identifies target regions where the two legs of the pedestrian are located respectively; input the sample image into an original neural network model to obtain second label information of the output sample image; and adjust parameter values of each parameter of the original neural network model according to the first label information and the second label information to obtain the trained neural network model.

15. An electronic device, characterized in that, It includes: A processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory stores a computer program, and when the program is executed by the processor, the processor executes the steps of the method for detecting a pedestrian climbing over a traffic guardrail according to any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, It stores a computer program executable by the processor, and when the program runs on the processor, the processor executes the steps of the method for detecting a pedestrian climbing over a traffic guardrail according to any one of claims 1-7.

Images