Warehouse scene vehicle safety distance recognition method, electronic device and storage medium

Abstract

The application provides a warehouse scene vehicle and pedestrian safety distance recognition method, electronic equipment and storage medium, wherein, key frames are periodically extracted from warehouse operation real-time monitoring video data; a pre-trained target detection model is used to detect human targets and vehicle targets in the key frames; whether the detected vehicle targets are moving is determined by taking a plurality of continuous key frames, and if so, the next step is executed; the distance between the vehicle target and the human target in the last frame and the subsequent frame of the plurality of continuous key frames is calculated, and if the distance meets a preset condition, the vehicle and pedestrian distance is safe, otherwise, it is not safe. The application can accurately identify whether the distance between the vehicle and the pedestrian in the warehouse operation scene is safe without additional manpower and additional equipment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of warehouse scene safety, and particularly relates to a warehouse scene human-vehicle safety distance identification method, an electronic device and a storage medium. BACKGROUND

[0002] In the daily warehouse operation process, there are two high-frequency scenarios. One scenario is that large goods are transported into the warehouse and the goods are operated according to the specified area and specified stacking specification, which is completed by human and vehicle cooperation. The other scenario is that the warehouse inventory goods are transported out of the warehouse, which is also completed by human and vehicle cooperation. On the outdoor road, the rules of human and vehicle are restricted by traffic rules, such as the vehicle actively waiting for the pedestrian to cross the road, and the person is not allowed to shuttle randomly in the road. However, in the warehouse indoor scene, the work area of the worker is large and needs to shuttle back and forth, so it is necessary to identify the safety distance of the human and the vehicle, so that the corresponding prompt and warning can be performed according to the identification result.

[0003] If the safety distance is identified in the way of human monitoring, it will consume additional manpower, causing waste and manpower redundancy, or the vehicle and the person each carry a thing connection sensing device, so that the distance information of each other can be sensed, but the thing connection sensing device needs to be purchased additionally. SUMMARY

[0004] Therefore, in view of the above technical problems, a warehouse scene human-vehicle safety distance identification method, an electronic device and a storage medium are provided.

[0005] The technical scheme adopted by the present application is as follows:

[0006] On the one hand, a warehouse scene human-vehicle safety distance identification method is provided, comprising:

[0007] S101, periodically extracting key frames from warehouse operation real-time monitoring video data;

[0008] S102, detecting human targets and vehicle targets in the key frames through a pre-trained target detection model;

[0009] S103, taking a plurality of continuous key frames to determine whether the detected vehicle target is moving, if so, the next step is executed;

[0010] S104, calculating the distance between the vehicle target and the human target in the last frame and the subsequent frame of the plurality of continuous key frames, if the distance meets the preset condition, the human-vehicle distance is safe, otherwise, it is not safe.

[0011] In another aspect, an electronic device is provided, including a storage module including instructions loaded and executed by a processor, the instructions, when executed, causing the processor to perform the warehouse scene human-vehicle safety distance identification method described above.

[0012] In still another aspect, a computer-readable storage medium is provided, storing one or more programs, the one or more programs, when executed by a processor, implementing the warehouse scene human-vehicle safety distance identification method described above.

[0013] The present application can accurately identify whether the distance between the human and the vehicle in the warehouse operation scene is safe without the need for additional manpower and additional equipment. BRIEF DESCRIPTION OF DRAWINGS

[0014] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0015] Figure 1 The flowchart of the present application;

[0016] Figure 2 The schematic diagram of the present application. DETAILED DESCRIPTION

[0017] As Figure 1 shown, the present specification embodiment provides a warehouse scene human-vehicle safety distance identification method, including:

[0018] S101, periodically extracting key frames from warehouse operation real-time monitoring video data.

[0019] In the present embodiment, the period of extracting key frames is 1 frame per second.

[0020] S102, detecting human targets and vehicle targets in the key frames by a pre-trained target detection model.

[0021] Wherein, the target detection model is trained by sample pictures, and the extraction process of the sample pictures includes:

[0022] First, collect warehouse operation historical monitoring video data, then extract key frames from the collected warehouse operation historical monitoring video data, and then clean all the extracted key frames, which means deleting repeated pictures, flower screen pictures and green screen pictures, and labeling human targets and vehicle targets in each key frame after cleaning, the vehicle targets include motor vehicle targets, crane targets, forklift targets, etc.

[0023] S103, taking a plurality of continuous key frames to determine whether the detected vehicle target is moving, if so, then execute the next step.

[0024] After the human target and the vehicle target in the key frame are detected by the target detection model, if the human target and the vehicle target are detected, each target has a detection box, and based on this, the moving distance of the vehicle target in a plurality of continuous key frames (for example, 2 continuous key frames) is calculated according to the detection box information (representing the position of the detection box in the key frame) of the vehicle target in the plurality of continuous key frames, and if the moving distance is greater than or equal to a threshold value, the vehicle target is moving.

[0025] Taking 2 continuous key frames as an example, the distance that the vehicle moves in 2 seconds at a normal operation moving speed in the warehouse operation site can be taken as the threshold value.

[0026] When the vehicle is in a moving state, if the center points of the target boxes of the vehicle in any two subsequent continuous frames do not change, the moving state of the vehicle is modified to a stationary state at this time, and after the stationary state, it is not necessary to continue to judge whether the distance between the vehicle and the human is safe.

[0027] In S104, the distance between the vehicle target and the human target in the last frame and the subsequent frame of the plurality of continuous key frames is calculated, and if the distance meets a preset condition, the distance between the vehicle and the human is safe, otherwise, the distance between the vehicle and the human is not safe.

[0028] Some parameters of the multi-camera can be used to calculate the distance between the vehicle target and the human target in the video picture. However, for a monocular camera, these parameters are lacking, so the distance between the vehicle target and the human target in the video picture of the monocular camera cannot be calculated. In order to solve this problem, the length and width of the warehouse operation site are measured in advance, and grid marks are set, such as marking the four corner points of the grid. Here, the key frame is a grid picture, which is divided into blocks in a square grid with a length of 3 meters and a width of 3 meters. The advantage of the square grid is that it can reduce the complexity of the later calculation, and a rectangular grid can also be used.

[0029] Based on the grid key frame, the distance between the vehicle and the human can be judged by using the grid where the human target is located and the grid where the vehicle target is located, as shown in Figure 2 :

[0030] a. Determine the grid where the human target is located and the grid where the vehicle target is located.

[0031] Since the camera is often arranged obliquely downward, the grid where the lower limb part of the human target is located is taken as the grid where the human target is located, then the corner point D min closest to the lower limb part among the four corner points of the vehicle target (the four corner points of the vehicle target detection box) is determined, and finally, the grid where the corner point D min is located is taken as the grid where the vehicle target is located. In this way, the determination of the grid where the human target is located and the grid where the vehicle target is located is more accurate for the judgment of the distance between the vehicle and the human.

[0032] In this embodiment, as shown in Figure 2As shown, the lower limb of the human target is represented by the midpoint D0 of the lower 1 / 3 of the human target (the lower 1 / 3 of the human target detection box). Of course, other points can also be used to represent the lower limb of the human target.

[0033] b. If the grid containing the person and the grid containing the vehicle are adjacent, the distance between them is unsafe; otherwise, it is safe.

[0034] In real-world scenarios, although the grid containing a person may be adjacent to the grid containing a vehicle, either the person or the vehicle may be located at the edge of its grid. Therefore, using grid adjacency alone as the sole basis for determining whether the distance between a person and a vehicle is unsafe may be inaccurate. Thus, when the grid containing a person and the grid containing a vehicle are adjacent, further consideration is needed:

[0035] a. Calculate the pixel distance d1 between the lower limb and the nearest corner point.

[0036] In this embodiment, d1 is D0 and D min The pixel distance.

[0037] b. If d1≤d2, then the distance between people and vehicles is unsafe; otherwise, the distance between people and vehicles is safe.

[0038] Specifically, if the grid containing the person target is diagonally adjacent to the grid containing the vehicle target, then d2 is the vertical side length of the grid containing the person target. If the grid containing the person target is horizontally adjacent to the grid containing the vehicle target, then d2 is the horizontal side length of the grid containing the person target. If the grid containing the person target is vertically adjacent to the grid containing the vehicle target, then d2 is the vertical side length of the grid containing the person target.

[0039] Based on the same inventive concept, embodiments of this specification also provide an electronic device, including a storage module, the storage module including instructions loaded and executed by a processor, the instructions, when executed, causing the processor to perform the steps of various exemplary embodiments of the present invention described in the above-described section on a method for recognizing safe distances between people and vehicles in a warehouse scenario.

[0040] The storage module may include readable media in the form of volatile storage cells, such as random access memory (RAM) and / or cache memory cells, and may further include read-only memory (ROM).

[0041] Based on the same inventive concept, embodiments of this specification also provide a computer-readable storage medium that stores one or more programs, which, when executed by a processor, implement the steps described in the above-described section of this specification regarding a method for identifying safe distances between people and vehicles in a warehouse scenario, according to various exemplary embodiments of the present invention.

[0042] Computer-readable storage media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples of computer-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0043] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0044] However, those skilled in the art should recognize that the above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Any changes or modifications to the above embodiments that are within the essential spirit of the present invention will fall within the scope of the claims of the present invention.

Claims

1. A method for recognizing safe distances between people and vehicles in a warehouse setting, characterized in that, include: S101. Periodically extract key frames from real-time monitoring video data of warehousing operations. The warehousing operation site is pre-marked with grid markers. The key frame is a gridded image. The warehousing operation monitoring camera is a monocular camera, which is tilted downwards. S102. Detect human and vehicle targets in the keyframes using a pre-trained target detection model; S103. Take multiple consecutive keyframes to determine whether the detected vehicle target is moving. If so, proceed to the next step. S104. Calculate the distance between the vehicle target and the person target in the last frame and subsequent frames of the plurality of consecutive keyframes. If the distance meets a preset condition, the distance between the vehicle and the person is safe; otherwise, it is unsafe. Determine the grid containing the human target and the grid containing the vehicle target: Use the grid containing the lower limbs of the human target as the grid containing the human target; determine the corner point among the four corner points of the vehicle target that is closest to the lower limbs; use the grid containing the closest corner point as the grid containing the vehicle target; If the grid containing the person target is adjacent to the grid containing the vehicle target, the distance between the person and the vehicle is unsafe; otherwise, it is safe.

2. The method for recognizing safe distances between people and vehicles in a warehouse scenario according to claim 1, characterized in that, The keyframe extraction period is 1 frame per second.

3. The method for recognizing safe distances between people and vehicles in a warehouse scenario according to claim 1, characterized in that, The target detection model is trained using sample images, and the process of extracting the sample images includes: Collect historical monitoring video data of warehousing operations; Extract keyframes from the historical monitoring video data of the warehousing operations; All extracted keyframes are cleaned, and human and vehicle targets are labeled in each cleaned keyframe.

4. The method for recognizing safe distances between people and vehicles in a warehouse setting according to claim 3, characterized in that, The step of cleaning the extracted keyframes further includes: Delete duplicate images, images with distorted screens, and photos with green screens.

5. The method for recognizing safe distances between people and vehicles in a warehouse scenario according to claim 2, characterized in that, The step of determining whether the detected vehicle target is moving by taking multiple consecutive keyframes further includes: Based on the detection box information of the vehicle target in the multiple consecutive keyframes, calculate the movement distance of the vehicle target in the multiple consecutive keyframes; If the moving distance is greater than or equal to the threshold, then the vehicle target is moving.

6. The method for recognizing safe distances between people and vehicles in a warehouse scenario according to claim 5, characterized in that, The multiple consecutive keyframes are two consecutive keyframes.

7. The method for recognizing safe distances between people and vehicles in a warehouse scenario according to claim 1, characterized in that, The lower limb portion of the human target is represented by the midpoint of the lower 1 / 3 of the target.

8. The method for recognizing safe distances between people and vehicles in a warehouse scenario according to claim 1, characterized in that, The statement that if the grid where the person target is located is adjacent to the grid where the vehicle target is located, the distance between the person and the vehicle is unsafe further includes: Calculate the pixel distance d1 between the lower limb portion and the nearest corner point; If d1≤d2, then the distance between people and vehicles is unsafe; otherwise, the distance between people and vehicles is safe. Wherein, if the grid where the person target is located is diagonally adjacent to the grid where the vehicle target is located, then d2 is the vertical side length of the grid where the person target is located; if the grid where the person target is located is horizontally adjacent to the grid where the vehicle target is located, then d2 is the horizontal side length of the grid where the person target is located; if the grid where the person target is located is vertically adjacent to the grid where the vehicle target is located, then d2 is the vertical side length of the grid where the person target is located.

9. An electronic device, characterized in that, The system includes a storage module, which includes instructions loaded and executed by a processor, which, when executed, cause the processor to perform a method for identifying safe distances between people and vehicles in a warehouse scenario according to any one of claims 1-8.

10. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by the processor, they implement the method for recognizing safe distances between people and vehicles in a warehouse scenario as described in any one of claims 1-8.

Images