Pedestrian crossing alarm method, device, equipment and system

By using pre-trained portrait segmentation models in the logistics industry to detect video streams, and judge the cross-border behavior of pedestrians around the logistics conveyor belt, the problems of missed and false alarms in the existing technology are solved, and more flexible and accurate detection is achieved , improves the efficiency of safety supervision.

CN112001336BActive Publication Date: 2024-11-26SHANGHAI ZHONGTONGJI NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010875998.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-25
Publication Date
2024-11-26
Estimated Expiration
2040-08-25

AI Technical Summary

Technical Problem

In the logistics industry, the existing technology has problems of missed and false alarms in the cross-border detection of pedestrians, and the detection is inflexible, making it difficult to adapt to the complex logistics working environment.

Method used

Use a pre-trained portrait segmentation model to preprocess the single-frame images in the video stream, determine the alarm area type, and judge whether pedestrians cross the boundary based on the portrait coordinates and segmentation diagrams, and issue alarm information, including horizontal, vertical and oblique Detection of line alarm areas.

Benefits of technology

It improves detection accuracy, reduces missed and false alarms, and can accurately determine pedestrian cross-border behavior in a complex logistics environment, saves human resources, and improves the efficiency of safety supervision.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112001336B_ABST
    Figure CN112001336B_ABST
Patent Text Reader

Abstract

The present application relates to a pedestrian crossing alarm method, device, equipment and system. The pedestrian crossing alarm method includes: obtaining a single frame image from a video stream at a preset time interval and preprocessing the single frame image; inputting the preprocessed single frame image into a pre-trained portrait segmentation model to obtain portrait coordinates and a portrait segmentation map; determining the type of the current alarm area, the types of alarm areas include horizontal alarm areas, vertical alarm areas and oblique alarm areas; judging whether there is a pedestrian crossing the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rules corresponding to the type of the current alarm area, and if so, issuing an alarm message. In this way, the detection is more flexible, and a more accurate judgment can be made on pedestrian crossing behaviors around the logistics conveyor belt in a complex and changeable logistics working environment, avoiding the occurrence of pedestrian crossing alarm omissions and false alarms, and saving human resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of logistics safety detection technology, and specifically to a pedestrian crossing alarm method, device, equipment and system. Background Art

[0002] With the rapid development of the current logistics industry and the continuous innovation of enterprise management, 6S on-site management is increasingly valued by logistics companies and plays an increasingly important role in improving management levels. In 6S on-site management, safety is the foundation, and personal safety is particularly important. In order to ensure the personal safety of operators in logistics sorting centers, all logistics companies have safety regulations that prohibit operators from directly crossing conveyor belts (including standing or sitting on conveyor belts).

[0003] In the relevant technology, there are two ways to supervise the pedestrian crossing the boundary in the safety regulations of logistics companies. One is manual supervision, but the manual supervision method requires people to be alert at all times, and it is difficult to make a comprehensive and timely alarm for pedestrian crossing the boundary; the other is to realize the alarm of pedestrian crossing the boundary by intelligently judging whether there is a human body in the monitoring area, but this method mostly uses traditional machine learning algorithms, which detect the human body or a part of the human body, and then compare the position of the detected part with the set alarm area. Missed reports and false alarms often occur, and the detection is not flexible, and there are great restrictions on the alarm area, which is difficult to adapt to the complex environment of logistics work. Summary of the invention

[0004] In view of this, the purpose of the present application is to overcome the deficiencies of the prior art and provide a pedestrian crossing alarm method, device, equipment and system.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] The first aspect of the present application provides a pedestrian crossing alarm method, comprising:

[0007] Acquire a single frame image from the video stream at a preset time interval, and preprocess the single frame image;

[0008] Input the preprocessed single-frame image into the pre-trained portrait segmentation model to obtain the portrait coordinates and portrait segmentation map;

[0009] Determine the type of the current alarm area, the types of the alarm area include horizontal alarm area, vertical alarm area and oblique alarm area;

[0010] According to the portrait coordinates, the portrait segmentation map and the alarm rules corresponding to the type of the current alarm area, it is determined whether there is a pedestrian crossing the boundary of the current alarm area, and if so, an alarm message is issued.

[0011] Optionally, judging whether a pedestrian crosses the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rule corresponding to the type of the current alarm area includes:

[0012] For each of the portrait coordinates, the following operations are performed respectively: according to the current portrait coordinates, whether the short side length of the current pedestrian frame is greater than a first threshold value is determined; if so, a first percentage of the overlapping area of ​​the current pedestrian frame and the current alarm area to the area of ​​the current pedestrian frame is calculated, and whether the first percentage is greater than a second threshold value is determined; if so, a human body segmentation area corresponding to the current portrait coordinates is determined according to the portrait segmentation map, and a second percentage of the overlapping area of ​​the human body segmentation area and the current alarm area to the area of ​​the human body segmentation area is calculated, and whether the second percentage is greater than a third threshold value is determined; if so, the current portrait coordinates meet the first preset condition;

[0013] Determine whether there is a portrait coordinate that meets the first preset condition among the portrait coordinates;

[0014] If so, for each of the portrait coordinates that meets the first preset condition, respectively perform the following operations: determine whether the current portrait coordinates meet the alarm rule corresponding to the type of the current alarm area; if so, the current portrait coordinates meet the second preset condition;

[0015] It is determined whether there are portrait coordinates that meet the second preset condition. If so, a pedestrian has crossed the boundary of the current alarm area; if not, no pedestrian has crossed the boundary of the current alarm area.

[0016] Optionally, the alarm rule corresponding to the horizontal alarm area includes: the lower left corner coordinates and the lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0017] Optionally, the alarm rule corresponding to the vertical alarm area includes: the upper left corner and lower left corner coordinates or the upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0018] Optionally, the alarm rule corresponding to the oblique line alarm area includes: at least two coordinate points among the upper left corner, lower left corner, upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0019] Optionally, the method further includes:

[0020] Collect several original images and construct a sample training set;

[0021] The pre-built deep learning network model is trained using the sample training set to obtain the portrait segmentation model.

[0022] Optionally, the issuing of alarm information includes:

[0023] Issue warnings of danger of crossing the boundary by voice and / or text.

[0024] The second aspect of the present application provides a pedestrian crossing alarm device, comprising:

[0025] An acquisition module, used to acquire a single frame image from a video stream at a preset time interval and pre-process the single frame image;

[0026] A recognition module is used to input the preprocessed single-frame image into a pre-trained portrait segmentation model to obtain portrait coordinates and a portrait segmentation map;

[0027] A determination module, used to determine the type of the current alarm area, wherein the types of the alarm area include a horizontal alarm area, a vertical alarm area, and a diagonal alarm area;

[0028] The alarm module is used to determine whether there is a pedestrian crossing the boundary of the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rules corresponding to the type of the current alarm area, and if so, issue an alarm message.

[0029] The third aspect of the present application provides a pedestrian crossing alarm device, comprising:

[0030] A processor, and a memory connected to the processor;

[0031] The memory is used to store computer programs;

[0032] The processor is used to call and execute the computer program in the memory to perform the method as described in the first aspect of the present application.

[0033] The fourth aspect of the present application provides a pedestrian crossing alarm system, comprising a camera and a pedestrian crossing alarm device as described in the third aspect of the present application connected to the camera; the camera is used to provide a video stream.

[0034] The technical solution provided by this application may have the following beneficial effects:

[0035] In the scheme of the present application, according to the preset time interval, a single frame image can be intercepted from the video stream for detection to realize the supervision of pedestrian crossing the boundary in the monitoring area, and the pre-trained portrait segmentation model is used to identify the pre-processed single frame image. Different from the method of detecting pedestrians in the monitoring area in the prior art, the idea of ​​portrait segmentation can greatly improve the detection accuracy; after obtaining the portrait coordinates and the portrait segmentation model, the type of the current alarm area is determined from the horizontal alarm area, the vertical alarm area and the oblique alarm area that are more suitable for the logistics working environment, and then according to the type of the current alarm area, the portrait coordinates and the portrait segmentation map, it is determined whether the current alarm area needs to be alarmed for pedestrian crossing the boundary, and after determining that the pedestrian crossing the boundary alarm is needed, the alarm information is issued. In this way, the detection is more flexible, and the pedestrian crossing behavior occurring around the logistics conveyor belt in the complex and changeable logistics working environment can be more accurately determined, avoiding the occurrence of pedestrian crossing alarm omissions and false alarms, saving human resources, and bringing convenience to the safety inspectors in the logistics industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 This is a flowchart of a pedestrian crossing alarm method provided by an embodiment of the present application.

[0038] Figure 2 It is a structural diagram of a network input end in a portrait segmentation model provided in another embodiment of the present application.

[0039] Figure 3 This is a schematic diagram of a branch structure of a region recommendation network in a portrait segmentation model provided by another embodiment of the present application.

[0040] Figure 4 This is a schematic diagram of a pedestrian frame and a human body segmentation area in a single frame image provided by another embodiment of the present application.

[0041] Figure 5 It is a structural schematic diagram of a pedestrian crossing alarm device provided in another embodiment of the present application.

[0042] Figure 6 It is a structural schematic diagram of a pedestrian crossing alarm device provided in another embodiment of the present application.

[0043] Figure 7It is a structural diagram of a pedestrian crossing alarm system provided by another embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other implementation methods obtained by ordinary technicians in the field without creative work belong to the scope of protection of the present application.

[0045] This embodiment provides a pedestrian crossing alarm method, which may specifically include the following steps:

[0046] Step 11: Obtain a single frame image from the video stream at a preset time interval, and pre-process the single frame image.

[0047] Step 12: Input the preprocessed single-frame image into a pre-trained portrait segmentation model to obtain portrait coordinates and a portrait segmentation map.

[0048] The portrait coordinates are the center point coordinates and width and height of the pedestrian frame (a rectangular frame representing the human body); the portrait segmentation map is a map showing the segmented area of ​​the human body.

[0049] Step 13: Determine the type of the current alarm area. The types of alarm areas include horizontal alarm areas, vertical alarm areas, and oblique alarm areas.

[0050] Among them, the type of alarm area can be determined according to the direction of the conveyor belt in the alarm area. For example, in the field of view of the alarm area captured by the camera, if the logistics conveyor belt is in a horizontal state, the type of the alarm area is a horizontal alarm area; if the logistics conveyor belt is in a vertical state, the type of the alarm area is a vertical alarm area; if the logistics conveyor belt is in a diagonal state, the type of the alarm area is a diagonal alarm area.

[0051] Step 14: According to the portrait coordinates, the portrait segmentation map and the alarm rules corresponding to the type of the current alarm area, it is determined whether there is a pedestrian crossing the current alarm area. If so, an alarm message is issued.

[0052] In this embodiment, according to the preset time interval, a single frame image can be intercepted from the video stream for detection to realize the supervision of pedestrian crossing the boundary in the monitoring area, and the pre-trained portrait segmentation model is used to identify the pre-processed single frame image. Different from the method of detecting pedestrians in the monitoring area in the prior art, the idea of ​​portrait segmentation can greatly improve the detection accuracy; after obtaining the portrait coordinates and the portrait segmentation model, the type of the current alarm area is determined from the horizontal alarm area, the vertical alarm area and the oblique alarm area that are more suitable for the logistics working environment, and then according to the type of the current alarm area, the portrait coordinates and the portrait segmentation map, it is determined whether the current alarm area needs to be alarmed for pedestrian crossing the boundary, and after determining that the pedestrian crossing the boundary alarm is needed, the alarm information is issued. In this way, the detection is more flexible, and the pedestrian crossing behavior occurring around the logistics conveyor belt in the complex and changeable logistics working environment can be more accurately determined, avoiding the occurrence of pedestrian crossing alarm omissions and false alarms, saving human resources, and bringing convenience to the safety inspectors in the logistics industry.

[0053] In specific implementation, the video stream can be accessed according to the rtsp address of the camera of the operation center, and a single frame image can be captured from it at a preset time interval, for example, a single frame image can be captured every 30 seconds, so as to timely understand the operation behavior of pedestrians and avoid untimely alarm when crossing the boundary. The captured single frame image is used as the image to be identified, and the image is preprocessed. The preprocessing operation may include a normalization operation. The specific operation process may include the following steps: first, the image to be identified is scaled into an RGB three-channel image with a length and width of 224, and then the mean and variance are subtracted for each of the three channels of the image. Finally, all pixels in the image are normalized to the interval [-1,1] to obtain the preprocessed image.

[0054] The preset time interval can be set according to actual needs and is not limited here.

[0055] The preprocessed image is input into the pre-trained portrait segmentation model for recognition. The specific recognition process can be divided into the following parts according to the processing order:

[0056] 1) Network input stem

[0057] like Figure 2As shown, the network input includes a convolution layer 201, a normalization layer 202, an activation function 203 and a maximum pooling layer 204. The preprocessed image entering the network input first passes through a two-dimensional convolution layer (CONV2D). A 7*7 convolution kernel extracts features from an input image of size 224*224*3 to reduce the overall computational complexity of the network. Since the step size of the convolution kernel is 2, the length and width of the obtained feature map are reduced to 1 / 2 of the original input image. Then, a batch normalization layer is used to normalize the feature map, which accelerates training and increases nonlinear fitting capabilities. Then, an activation function (ReLU) is used. Finally, a maximum pooling layer is used to further reduce the resolution of the feature map. At this time, the size of the feature map is reduced to 1 / 4 of the original image, that is, the size of the output feature map is 56*56*64.

[0058] 2) Extract feature network

[0059] The essence of this feature extraction network is to use residual blocks to extract features from 56*56*64 feature maps, and finally output 14*14*1024 feature maps. Using residual blocks for feature extraction can be divided into three stages. All three stages have the same purpose, which is to expand the number of channels of the input feature map to twice the original number, while reducing the length and width to 1 / 2 of the original number. The basic components are downsampling residual blocks and identity residual blocks. Each of the three stages consists of a downsampling residual block and several identity residual blocks. Among them, downsampling will set the initial convolution step size to 2, and in this way, the feature map is downsampled to reduce the length and width. In the identity residual block, by setting the convolution-related parameters, the input and output feature map sizes of the residual block can be controlled to be consistent, so as to perform addition processing and avoid the gradient disappearance and degradation problems of the deep network.

[0060] 3) Region Proposal Networks (RPN)

[0061] The subnetwork inputs a 14*14*1024 feature map and outputs several proposal boxes, each of which corresponds to 4 regression coordinate values ​​(coordinates / object proposals) and 2 classification scores (objectness score). Figure 3As shown in the figure, the subnetwork is actually divided into two branches. One branch is the reglayer in the figure, which is used to predict the center coordinates (x, y) and the width and height (w, h) of the recommendation box. Each recommendation box has 4 values. The other branch is the cls layer in the figure, which is used to determine whether the recommendation box is foreground or background. It can be obtained by binary classification using softmax. In addition, the anchor in the figure refers to the anchor point, which can be understood as the mapping point of the center of the sliding window on the input single frame image. When used, this anchor will be used as the center to generate k different recommendation boxes. In actual use, k = 9, that is, each sliding window has 3 different sizes and 3 different aspect ratios.

[0062] 4)ROI pooling part

[0063] The input of this part is the 14*14*1024 feature map output by the feature extraction network and several recommended boxes output by the regional recommendation network. ROI pooling is a simplified form of SPP-Layer. Roi refers to the rectangular box (n, x, y, h, w) output by RPN, n represents the number of roi, (x, y, h, w) is the position and size of roi in the original input image, and the coordinates (x, y, h, w) are mapped to the feature map to obtain the position and size (x', y', h', w') on the feature map. This window is divided into H*W sub-windows, and the size of each sub-window is approximately (h' / H)*(w' / W). For each sub-window, remove its maximum value, and finally get the output of ROIpooling, which is a 7*7*1024 feature map.

[0064] 5) Output part

[0065] The input of this part is the 7*7*1024 feature map output by the ROI pooling part, and the final output is the portrait coordinates and portrait segmentation map. The portrait coordinates are the coordinates of the center point and the width and height of the pedestrian frame; the portrait segmentation map is a 14*14*2 feature map. One 14*14 feature map represents the background, and the other 14*14 feature map represents the pedestrian. In addition, the feature map representing the pedestrian is processed. The pixel value on the feature map is a floating point number between 0 and 1, indicating the probability that the pixel is a pedestrian. We can set a threshold, set the pixel value greater than the threshold to 1, and the pixel value less than the threshold to 0. If it is 1, it means that the pixel represents a pedestrian, otherwise it is 0, which means that the pixel is the background.

[0066] Among them, the training method of the portrait segmentation model may include: collecting a number of original images to construct a sample training set; using the sample training set to train a pre-constructed deep learning network model to obtain a portrait segmentation model.

[0067] Specifically, the specific implementation method of the portrait segmentation model can refer to the existing related technology and will not be repeated here.

[0068] After obtaining the portrait coordinates and portrait segmentation map of the input single-frame image through the recognition of the portrait segmentation model, determine which type of alarm area the current environment belongs to from the horizontal alarm area, vertical alarm area and diagonal alarm area, and manually mark the current alarm area in advance. Compare the obtained portrait coordinates and portrait segmentation map with the marked current alarm area, and determine whether the alarm area needs to issue a pedestrian crossing alarm through the alarm rules corresponding to the type of the current alarm area.

[0069] Among them, the position of the pedestrian frame in the input single frame image can be calculated based on the portrait coordinates. The pedestrian frame is essentially a rectangular frame representing the human body. The human body segmentation area can be obtained based on the portrait segmentation map. The human body segmentation area is essentially an area surrounding the human body. Figure 4 As shown, the rectangular frame 401 is the pedestrian frame, and the area 402 is the human body segmentation area.

[0070] In order to adapt to the complex logistics working environment and ensure the accuracy of the judgment, the pedestrian crossing judgment can be divided into two stages: the first stage is the basic judgment; the second stage is the specific judgment. Among them, for all types of alarm areas, a basic judgment must be made first. Only after the basic judgment determines that there is a pedestrian crossing behavior, can a specific judgment be made according to the type of alarm area. In this way, accurate detection of pedestrian crossing behaviors in different scenarios can be achieved, improving the work safety factor.

[0071] In the basic judgment stage, considering that there may be multiple portrait coordinates in a single frame image, the following operations can be performed for each portrait coordinate:

[0072] a. According to the current portrait coordinates, determine whether the short side length of the current pedestrian frame is greater than the first threshold. This method can filter out smaller portraits in a single-frame image to reduce interference. For example, people who are far away from the camera will appear smaller in the photo and can be filtered out. Only nearby portraits need to be judged.

[0073] b. If the short side of the current pedestrian frame is longer than the first threshold, then calculate the first percentage of the overlapping area of ​​the current pedestrian frame and the current alarm area to the area of ​​the current pedestrian frame, and determine whether the first percentage is greater than the second threshold. The calculation formula is as follows:

[0074]

[0075] Among them, A represents the pedestrian frame, B represents the current alarm area, S Arepresents the pedestrian box area (the rectangular box area representing the human body), S A∩B Indicates the overlapping area between the current pedestrian frame and the current alarm area.

[0076] c. If the first percentage is greater than the second threshold, similarly, the human body segmentation area corresponding to the current portrait coordinates can be determined based on the portrait segmentation map, and the second percentage of the overlapping area between the human body segmentation area and the current alarm area to the area of ​​the human body segmentation area can be calculated to determine whether the second percentage is greater than the third threshold; if so, the current portrait coordinates meet the first preset condition.

[0077] After determining the coordinates of each portrait, for the input single-frame image, determine whether there are portrait coordinates that meet the first preset condition among the portrait coordinates in the image, so as to determine whether there is pedestrian crossing behavior in the image in the basic determination stage.

[0078] If the input single-frame image contains portrait coordinates that meet the first preset conditions, it can be determined that there is a pedestrian crossing the boundary in the image; if the input single-frame image does not contain portrait coordinates that meet the first preset conditions, it can be determined that there is no pedestrian crossing the boundary in the image and no alarm is needed.

[0079] After determining that there is a pedestrian crossing the boundary in the basic judgment stage, it is necessary to enter the specific judgment stage. In the specific judgment stage, it is also considered that there may be multiple portrait coordinates that meet the first preset condition in a single frame image, so for each portrait coordinate that meets the first preset condition, the following operations can be performed respectively: determine whether the current portrait coordinates meet the alarm rules corresponding to the type of the current alarm area, and if so, the current portrait coordinates meet the second preset condition.

[0080] After judging each portrait coordinate that meets the first preset condition, for the input single-frame image, it is judged whether there are portrait coordinates that meet the second preset condition among each portrait coordinate that meets the first preset condition in the image, so as to determine whether there is a pedestrian crossing the boundary in the specific judgment stage of the image.

[0081] If the input single-frame image contains portrait coordinates that meet the second preset condition, it can be determined that a pedestrian has crossed the boundary of the current alarm area; if not, it can be determined that no pedestrian has crossed the boundary of the current alarm area.

[0082] In some embodiments, in order to increase the detection accuracy of pedestrian crossing behavior around the conveyor belt, corresponding alarm rules are set for different types of alarm areas. Among them, for the horizontal alarm area, it is generally believed that the coordinates of the lower left corner and the lower right corner of the pedestrian box mark the lower body of the person, and the corresponding alarm rules may include: the lower left corner coordinates and the lower right corner coordinates of the portrait coordinates are within the current alarm area. For the vertical alarm area, it is generally believed that the left side coordinates (upper left corner and lower left corner) or the right side coordinates (upper right corner and lower right corner) of the pedestrian box should be in the alarm area, and the corresponding alarm rules may include: the upper left corner and lower left corner coordinates or the upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area. Similarly, the alarm rules corresponding to the oblique alarm area may include: at least two coordinate points in the upper left corner, lower left corner, upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0083] After determining that a pedestrian has crossed the boundary in a single frame image, in the above step 14, issuing an alarm message may specifically include: issuing a warning of crossing the boundary in the form of sound and / or text, so as to promptly warn the pedestrian who has crossed the boundary that he is in danger, and remind the pedestrian to correct his behavior in time to avoid danger. For example, when a pedestrian has crossed the boundary, a voice prompt "crossing the boundary has occurred, please correct it in time" is issued until it is detected that there is no pedestrian crossing the boundary.

[0084] This embodiment provides a pedestrian crossing alarm device, such as Figure 5 As shown, the device may specifically include: an acquisition module 501, used to acquire a single-frame image from a video stream at a preset time interval and pre-process the single-frame image; an identification module 502, used to input the pre-processed single-frame image into a pre-trained portrait segmentation model to obtain portrait coordinates and a portrait segmentation map; a determination module 503, used to determine the type of the current alarm area, the types of alarm areas include horizontal alarm areas, vertical alarm areas and oblique alarm areas; an alarm module 504, used to determine whether there is a pedestrian crossing the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rules corresponding to the type of the current alarm area, and if so, issue an alarm message.

[0085] Optionally, the above-mentioned alarm rules corresponding to the portrait coordinates, the portrait segmentation map and the type of the current alarm area are used to judge whether there is a pedestrian crossing the current alarm area. The alarm module is specifically used to: for each portrait coordinate, perform the following operations respectively: according to the current portrait coordinates, judge whether the short side length of the current pedestrian frame is greater than a first threshold; if so, calculate the first percentage of the overlapping area of ​​the current pedestrian frame and the current alarm area to the area of ​​the current pedestrian frame, and judge whether the first percentage is greater than a second threshold; if so, determine the human body segmentation area corresponding to the current portrait coordinates according to the portrait segmentation map, and calculate the overlapping area of ​​the human body segmentation area with the current alarm area. The second percentage of the area of ​​the human body segmentation area is used to determine whether the second percentage is greater than the third threshold value; if so, the current portrait coordinates meet the first preset condition; it is determined whether there are portrait coordinates that meet the first preset condition among the portrait coordinates; if so, for each portrait coordinate that meets the first preset condition, the following operations are performed respectively: it is determined whether the current portrait coordinates meet the alarm rules corresponding to the type of the current alarm area, if so, the current portrait coordinates meet the second preset condition; it is determined whether there are portrait coordinates that meet the second preset condition, if so, there is a pedestrian crossing the boundary in the current alarm area; if not, there is no pedestrian crossing the boundary in the current alarm area.

[0086] Optionally, the alarm rule corresponding to the horizontal alarm area may specifically include: the lower left corner coordinates and the lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0087] Optionally, the alarm rule corresponding to the vertical alarm area may specifically include: the upper left corner and lower left corner coordinates or the upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0088] Optionally, the alarm rule corresponding to the above-mentioned oblique line alarm area may specifically include: at least two coordinate points among the upper left corner, lower left corner, upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area.

[0089] Optionally, the pedestrian crossing alarm device may further include a training module, which may be specifically used to: collect a number of original images to construct a sample training set; and use the sample training set to train a pre-constructed deep learning network model to obtain a portrait segmentation model.

[0090] Optionally, the alarm module for sending out the alarm information may be specifically used to send out a warning of the danger of crossing the boundary in the form of sound and / or text.

[0091] Specifically, the specific implementation scheme of the pedestrian crossing alarm device provided in the embodiment of the present application can refer to the implementation scheme of the pedestrian crossing alarm method described in any of the above embodiments, which will not be repeated here.

[0092] The embodiment of the present application provides a pedestrian crossing alarm device, such as Figure 6 As shown, the device may include: a processor 601, and a memory 602 connected to the processor 601; the memory 602 is used to store computer programs; the processor 601 is used to call and execute the computer program in the memory 602 to perform the method described in any of the above embodiments.

[0093] Among them, the specific implementation scheme of the pedestrian crossing alarm device provided in the embodiment of the present application can refer to the implementation scheme of the pedestrian crossing alarm method described in any of the above embodiments, which will not be repeated here.

[0094] The embodiment of the present application provides a pedestrian crossing warning system, such as Figure 7 As shown, the system may include a camera 701 and a pedestrian crossing warning device 702 as described in any of the above embodiments connected to the camera 701; the camera 701 is used to provide a video stream.

[0095] Specifically, the specific implementation scheme of the pedestrian crossing alarm system provided in the embodiment of the present application can refer to the implementation scheme of the pedestrian crossing alarm method described in any of the above embodiments, which will not be repeated here.

[0096] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0097] It should be noted that, in the description of this application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0098] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0099] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0100] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0102] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0104] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A pedestrian crossing alarm method, characterized in that: include: Acquire a single frame image from the video stream at a preset time interval, and preprocess the single frame image; Input the preprocessed single-frame image into a pre-trained portrait segmentation model to obtain portrait coordinates and a portrait segmentation map; wherein the portrait coordinates are the center point coordinates and width and height of a pedestrian frame; the pedestrian frame is a rectangular frame surrounding a human body; Determine the type of the current alarm area, the types of the alarm area include horizontal alarm area, vertical alarm area and oblique alarm area; According to the portrait coordinates, the portrait segmentation map and the alarm rule corresponding to the type of the current alarm area, it is determined whether there is a pedestrian crossing the boundary of the current alarm area, and if so, an alarm message is issued; The step of judging whether a pedestrian has crossed the boundary of the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rule corresponding to the type of the current alarm area includes: For each of the portrait coordinates, the following operations are performed respectively: according to the current portrait coordinates, whether the short side length of the current pedestrian frame is greater than a first threshold value is determined; if so, a first percentage of the overlapping area of ​​the current pedestrian frame and the current alarm area to the area of ​​the current pedestrian frame is calculated, and whether the first percentage is greater than a second threshold value is determined; if so, a human body segmentation area corresponding to the current portrait coordinates is determined according to the portrait segmentation map, and a second percentage of the overlapping area of ​​the human body segmentation area and the current alarm area to the area of ​​the human body segmentation area is calculated, and whether the second percentage is greater than a third threshold value is determined; if so, the current portrait coordinates meet the first preset condition; Determine whether there is a portrait coordinate that meets the first preset condition among the portrait coordinates; If so, for each of the portrait coordinates that meets the first preset condition, respectively perform the following operations: determine whether the current portrait coordinates meet the alarm rule corresponding to the type of the current alarm area; if so, the current portrait coordinates meet the second preset condition; Determine whether there are portrait coordinates that meet the second preset condition, if yes, then there is a pedestrian crossing the current alarm area; if no, then there is no pedestrian crossing the current alarm area; The alarm rules corresponding to the horizontal alarm area include: the lower left corner coordinates and the lower right corner coordinates of the portrait coordinates are within the current alarm area; The alarm rules corresponding to the vertical alarm area include: the upper left corner and lower left corner coordinates or the upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area; The alarm rule corresponding to the oblique line alarm area includes: at least two coordinate points among the upper left corner, lower left corner, upper right corner and lower right corner of the portrait coordinates are within the current alarm area; The method further comprises: Collect several original images and construct a sample training set; Using the sample training set to train the pre-built deep learning network model to obtain the portrait segmentation model; The sending of alarm information includes: Issue warnings of danger of crossing the boundary by voice and / or text.

2. A pedestrian crossing alarm device, characterized in that: include: An acquisition module, used to acquire a single frame image from a video stream at a preset time interval and pre-process the single frame image; A recognition module is used to input the preprocessed single-frame image into a pre-trained portrait segmentation model to obtain portrait coordinates and a portrait segmentation map; The portrait coordinates are the center point coordinates and width and height of the pedestrian frame; the pedestrian frame is a rectangular frame surrounding the human body; A determination module, used to determine the type of the current alarm area, wherein the types of the alarm area include a horizontal alarm area, a vertical alarm area, and a diagonal alarm area; An alarm module, used to determine whether there is a pedestrian crossing the boundary of the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rule corresponding to the type of the current alarm area, and if so, issue an alarm message; The step of judging whether a pedestrian has crossed the boundary of the current alarm area according to the portrait coordinates, the portrait segmentation map and the alarm rule corresponding to the type of the current alarm area includes: For each of the portrait coordinates, the following operations are performed respectively: according to the current portrait coordinates, whether the short side length of the current pedestrian frame is greater than a first threshold value is determined; if so, a first percentage of the overlapping area of ​​the current pedestrian frame and the current alarm area to the area of ​​the current pedestrian frame is calculated, and whether the first percentage is greater than a second threshold value is determined; if so, a human body segmentation area corresponding to the current portrait coordinates is determined according to the portrait segmentation map, and a second percentage of the overlapping area of ​​the human body segmentation area and the current alarm area to the area of ​​the human body segmentation area is calculated, and whether the second percentage is greater than a third threshold value is determined; if so, the current portrait coordinates meet the first preset condition; Determine whether there is a portrait coordinate that meets the first preset condition among the portrait coordinates; If so, for each of the portrait coordinates that meets the first preset condition, respectively perform the following operations: determine whether the current portrait coordinates meet the alarm rule corresponding to the type of the current alarm area; if so, the current portrait coordinates meet the second preset condition; Determine whether there are portrait coordinates that meet the second preset condition, if yes, then there is a pedestrian crossing the current alarm area; if no, then there is no pedestrian crossing the current alarm area; The alarm rules corresponding to the horizontal alarm area include: the lower left corner coordinates and the lower right corner coordinates of the portrait coordinates are within the current alarm area; The alarm rules corresponding to the vertical alarm area include: the upper left corner and lower left corner coordinates or the upper right corner and lower right corner coordinates of the portrait coordinates are within the current alarm area; The alarm rule corresponding to the oblique line alarm area includes: at least two coordinate points among the upper left corner, lower left corner, upper right corner and lower right corner of the portrait coordinates are within the current alarm area; The pedestrian crossing alarm device is also used to collect a number of original images and construct a sample training set; Using the sample training set to train the pre-built deep learning network model to obtain the portrait segmentation model; The sending of alarm information includes: Issue warnings of danger of crossing the boundary by voice and / or text.

3. A pedestrian crossing alarm device, characterized in that: include: A processor, and a memory connected to the processor; The memory is used to store computer programs; The processor is configured to call and execute the computer program in the memory to perform the method according to claim 1.

4. A pedestrian crossing alarm system, characterized in that: It comprises a camera and a pedestrian crossing alarm device as claimed in claim 3 connected to the camera; the camera is used to provide a video stream.