Warehouse alarm method and device, electronic equipment and computer readable storage medium

Abstract

The warehouse alarm method and device, the electronic equipment and the computer readable storage medium provided by the application meet the same category recognition result of continuous multiple frames of pictures, and the difference between the time stamps corresponding to the adjacent two frames of pictures of the continuous multiple frames of pictures is less than the first threshold value, so that the recognition result of the continuous multiple frames of pictures is determined as a valid recognition result, the timeliness and accuracy of the recognition result are ensured, and then it is judged whether the staff wears a safety helmet based on the valid recognition result. Meanwhile, the state recognition of the warehouse operation equipment is improved, the misrecognition problem caused by the shaking of the recognition box is effectively avoided, and the false alarm rate is greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of warehouse alarm, and particularly relates to a warehouse alarm method and device, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] In a warehouse operation scenario, a worker is required to wear a safety helmet when in a dangerous area, and no one can be around a warehouse operation device when it is moving. When the above requirements are not met, timely alarm is needed to avoid accidents.

[0003] The existing warehouse alarm technology usually relies on target recognition results of a single picture, and cannot guarantee the timeliness and accuracy of recognition. When the recognition box shakes, the state of the warehouse operation device can be misrecognized, resulting in false alarms. SUMMARY

[0004] Therefore, in view of the above technical problems, a warehouse alarm method, device, electronic equipment and computer readable storage medium are provided.

[0005] The technical solutions adopted by the present application are as follows:

[0006] As a first aspect of the present application, a warehouse alarm method is provided, comprising:

[0007] real-time acquisition of a video in a warehouse;

[0008] identification of pictures of the video and acquisition of a current timestamp to obtain a recognition result, the recognition result including a recognized category, a recognition box coordinate and a corresponding timestamp, the recognized category including a human target, a human target wearing a safety helmet, a human target not wearing a safety helmet and a warehouse operation device target, wherein for the identification of each category, if the category recognition results of a plurality of consecutive pictures are the same, and the difference between the timestamps corresponding to the adjacent two pictures of the plurality of consecutive pictures is less than a first threshold, then the recognition results of the plurality of consecutive pictures are all valid recognition results;

[0009] determination of a real-time position of the human target according to the human target recognition box coordinate of the valid recognition result, and further determination of whether the human target is wearing a safety helmet if the real-time position is located in a preset dangerous area, and alarm processing if not;

[0010] real-time determination of the state of the warehouse operation device target: determination of whether the warehouse operation device target recognition box meets the following conditions according to the valid recognition results of the adjacent two pictures, and the warehouse operation device target is in a moving state if both conditions are met;

[0011] the center point offset of the warehouse operation device target recognition box of the adjacent two pictures exceeds a second threshold;

[0012] the overlap degree of the warehouse operation equipment target bounding boxes of the two adjacent frames of pictures is less than a second threshold value;

[0013] there are two pairs of diagonal points between the warehouse operation equipment target bounding boxes of the two adjacent frames of pictures, which satisfy a deviation condition, wherein the two points with the same position in the warehouse operation equipment target bounding boxes of the two adjacent frames of pictures constitute a pair of diagonal points, and the deviation condition is that the absolute value of the x coordinate difference or the absolute value of the y coordinate difference of the two diagonal points in a pair of diagonal points is greater than or equal to a third threshold value;

[0014] if the warehouse operation equipment target is in a motion state, it is further determined whether there is a person target in a preset range of the warehouse operation equipment target, and if so, an alarm processing is performed.

[0015] As a second aspect of the present application, a warehouse alarm device is provided, comprising:

[0016] a video acquisition module, configured to acquire videos in a warehouse in real time;

[0017] an identification module, configured to identify pictures of the videos and acquire a current timestamp to obtain an identification result, wherein the identification result comprises an identified category, bounding box coordinates and a corresponding timestamp, and the identified category comprises a person target, a person target wearing a safety helmet, a person target not wearing a safety helmet and a warehouse operation equipment target, wherein for the identification of each category, if the category identification results of a plurality of continuous frames of pictures are the same, and the difference between the timestamps corresponding to the two adjacent frames of pictures of the plurality of continuous frames of pictures is less than a first threshold value, then the identification results of the plurality of continuous frames of pictures are all valid identification results;

[0018] a first alarm module, configured to determine a real-time position of a person target according to the person target bounding box coordinates of the valid identification result, and if the real-time position is located in a preset dangerous area, it is further determined whether the person target is wearing a safety helmet, and if not, an alarm processing is performed;

[0019] a second alarm module, configured to determine the state of the warehouse operation equipment target in real time: according to the valid identification results of the two adjacent frames of pictures, it is determined whether the warehouse operation equipment target bounding boxes satisfy the following conditions, and if all the conditions are satisfied, the warehouse operation equipment target is in a motion state:

[0020] the center point offset of the warehouse operation equipment target bounding boxes of the two adjacent frames of pictures exceeds a second threshold value;

[0021] the overlap degree of the warehouse operation equipment target bounding boxes of the two adjacent frames of pictures is less than a second threshold value;

[0022] The two pairs of corner points between the warehouse operation equipment target identification boxes of the two adjacent frames of pictures meet the deviation condition, wherein the two points with the same position in the warehouse operation equipment target identification boxes of the two adjacent frames of pictures form a pair of corner points, and the deviation condition is that the absolute value of the x coordinate difference or the absolute value of the y coordinate difference of the two corner points in a pair of corner points is greater than or equal to a third threshold value;

[0023] If the warehouse operation equipment target is in a motion state, it is further determined whether there is a person target in a preset range of the warehouse operation equipment target, and if so, an alarm processing is performed.

[0024] As a third aspect of the present application, an electronic device is provided, comprising a storage module, the storage module comprising instructions loaded and executed by a processor, the instructions when executed causing the processor to perform the warehouse alarm method of the first aspect.

[0025] As a fourth aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium storing one or more programs, the one or more programs when executed by a processor implementing the warehouse alarm method of the first aspect.

[0026] In the present application, when performing target identification, the identification results of continuous multiple frames of pictures are determined as valid identification results only when the category identification results of the continuous multiple frames of pictures are the same, and the difference between the time stamps corresponding to the two adjacent frames of pictures of the continuous multiple frames of pictures is less than a first threshold value, thereby ensuring the timeliness and accuracy of the identification results, and further determining whether the staff wears a safety helmet based on the valid identification results, and improving the state identification of the warehouse operation equipment, effectively avoiding the misidentification problem caused by the shaking of the identification box, thereby greatly reducing the false alarm rate. BRIEF DESCRIPTION OF DRAWINGS

[0027] The present application will be described in detail below in conjunction with the drawings and specific embodiments:

[0028] Figure 1 A flowchart of a warehouse alarm method provided for the embodiments of the present application;

[0029] Figure 2 A motion state identification diagram of a warehouse operation equipment target of the embodiments of the present application;

[0030] Figure 3 A schematic diagram of a crane operation alarm device provided for the embodiments of the present application;

[0031] Figure 4 A schematic diagram of an electronic device provided for the embodiments of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application will be described below with reference to the accompanying drawings. It should be noted that the embodiments described in the specification are not exhaustive, and do not represent the only embodiment of the present application. The following embodiments are only for the purpose of clearly illustrating the inventive content of the present application, and are not a limitation on the embodiments. Based on the embodiment description, those skilled in the art can also make different forms of changes and modifications, and any changes or modifications that belong to the technical concept and inventive content of the present application and are obvious are within the protection scope of the present application.

[0033] As shown in Figure 1 The warehouse alarm method provided by the embodiment of the present application comprises:

[0034] S101, real-time acquisition of video in the warehouse.

[0035] S102, identification of pictures of the video by an AI target detection model, and acquisition of a current timestamp to obtain an identification result, the identification result comprising an identified category, an identified frame coordinate (two diagonal coordinates) and a corresponding timestamp, the identified category comprising a human target, a human target wearing a safety helmet, a human target not wearing a safety helmet and a warehouse operation equipment target, wherein for identification of each category, if the category identification results of continuous multiple pictures (such as 3 continuous pictures) are the same, and the difference between the timestamps corresponding to the adjacent two pictures of the continuous multiple pictures is less than a first threshold, then the identification results of the continuous multiple pictures are all valid identification results.

[0036] It should be noted that in the prior art, the identification result of a single picture is directly used, which cannot guarantee the timeliness of the result, and when the AI target detection is misidentified, the identification result is inaccurate. In the present application, the identification result of continuous multiple pictures is only considered as a valid identification result when the category identification results of the continuous multiple pictures are the same and the difference between the timestamps corresponding to the adjacent two pictures of the continuous multiple pictures is less than a first threshold, thereby guaranteeing the timeliness and accuracy of the identification result. For example, when the identification result is ABBAAA, obviously, the difference between the timestamp of the second A and the timestamp of the first A will be greater than the first threshold, then ABB is an invalid identification result, and AAA is a valid identification result, wherein A represents wearing a safety helmet and B represents not wearing a safety helmet.

[0037] The scene to which the method of the embodiment of the present application is applied usually has only one warehouse operation equipment, such as a forklift, and the human target usually has multiple, so in order to distinguish different human targets and the corresponding identification frame coordinates and timestamps, the different human targets are further tracked according to the identification result:

[0038] The recognition results of all person targets are input into a tracking model, and the ID of each person target is returned by the model, so that when a person target does not disappear, its ID will not change even if its position changes, such as person1, person2...personN.

[0039] Further, in order to correspond the person target wearing a safety helmet and the person target not wearing a safety helmet to different person targets, the recognition box of the person target wearing a safety helmet and the recognition box of the person target not wearing a safety helmet are determined to belong to the recognition box of the person target respectively through the recognition box coordinates, such as through calculation by recognition box coordinates, the recognition box a of a person target wearing a safety helmet is in the recognition box A of person1, and the recognition box a belongs to the recognition box A.

[0040] In the embodiment, the recognition results are saved through a structure:

[0041] Firstly, a structure is created, and the structure has members corresponding to categories one by one.

[0042] Then, the recognition box coordinates of the same category obtained by each identification and the corresponding time stamps are saved in the corresponding members in turn, such as:

[0043] ‘person1’:[{timestamp: recognition box coordinates},{{timestamp: recognition box coordinates},...]

[0044] ‘person1_DM’:[{timestamp: recognition box coordinates},{{timestamp: recognition box coordinates},...]

[0045] ‘person1_WDM’:[{timestamp: recognition box coordinates},{{timestamp: recognition box coordinates},...]

[0046] ‘person2’:[{timestamp: recognition box coordinates},{{timestamp: recognition box coordinates},...] ...

[0047] ‘folklift’:[{timestamp: recognition box coordinates},{{timestamp: recognition box coordinates},...]

[0048] person1_DM represents the person target 1 wearing a safety helmet, person1_WDM represents the person target 1 not wearing a safety helmet, and folklift represents a forklift.

[0049] Based on the above structure, if the difference between the currently saved time stamp and the previous time stamp in the same structure member is greater than or equal to a first threshold, all recognition box coordinates and corresponding time stamps before the currently saved time stamp in the member are deleted, such as:

[0050] person1_WDM: [{timestamp1: bounding box coordinates1}, {timestamp2: bounding box coordinates2}, {timestamp3: bounding box coordinates3}], if the difference between the currently saved timestamp3 and timestamp2 is greater than or equal to the first threshold, delete {timestamp1: bounding box coordinates1}, {timestamp2: bounding box coordinates2}, when there are three timestamps in the 'person1_WDM' member, that is, 'person1_WDM': [{timestamp3: bounding box coordinates3}, {timestamp4: bounding box coordinates4}, {timestamp5: bounding box coordinates5}], it means that the person target 1 wears a safety helmet in the three consecutive picture frames, and the difference between the timestamps corresponding to the adjacent two frames of the three consecutive picture frames is less than the first threshold. It can be seen that after such processing, each member corresponds to the set of valid recognition results of the corresponding category.

[0051] S103, determining the real-time position of the person target according to the valid recognition result of the person target bounding box coordinates, if the real-time position is located in the preset dangerous area, further determining whether the person target wears a safety helmet, if not, performing alarm processing.

[0052] S104, real-time determination of the state of the warehouse operation equipment target: according to the valid recognition results of the previous and next two adjacent picture frames, determining whether the warehouse operation equipment target bounding box satisfies the following conditions, if all the conditions are met, the warehouse operation equipment target is in a motion state:

[0053] Condition 1: the center point offset of the warehouse operation equipment target bounding box of the previous and next two adjacent picture frames exceeds the second threshold;

[0054] Condition 2: the coincidence degree of the warehouse operation equipment target bounding box of the previous and next two adjacent picture frames is less than the second threshold;

[0055] Condition 3: there are two pairs of corner points that satisfy the deviation condition between the warehouse operation equipment target bounding boxes of the previous and next two adjacent picture frames, wherein the two points with the same position in the warehouse operation equipment target bounding boxes of the previous and next two adjacent picture frames constitute a pair of corner points, and the deviation condition is met, that is, the absolute value of the x coordinate difference or the absolute value of the y coordinate difference of the two corner points in a pair of corner points is greater than or equal to the third threshold.

[0056] In the prior art, the target stillness and motion recognition is usually identified only using condition 1 and condition 2. When the warehouse operation equipment is working, the operator on the equipment (such as a forklift driver) is exposed in a high proportion in the video picture, so when the equipment identification is performed, the AI model will probably identify the object or person close to the equipment as part of the equipment, which leads to the fact that even if the equipment is parked and not working, the size of the identification frame in the adjacent picture frames before and after will change (frame jitter). Once the identification frame changes greatly, only using condition 1 and condition 2, the equipment will be misidentified as a motion state, and the application increases condition 3, which can effectively avoid misidentification, such as Figure 2 as shown in Figure 2 (a) In (a), the upper left corner points of the two identification frames coincide (the coordinate difference is 0, which is less than the third threshold value), and the lower left corner points coincide (the coordinate difference is 0, which is less than the third threshold value), and it is identified as a still state, Figure 2 (b) In (b), the four corner points of the two identification frames are far apart (the coordinate difference is greater than or equal to the third threshold value), and it is identified as a motion state, Figure 2 (c) In (c), the upper left corner points of the two identification frames coincide (the coordinate difference is 0, which is less than the third threshold value), and the lower left corner points are close (the coordinate difference is greater than 0, which is less than the third threshold value), and it is identified as a still state.

[0057] If the warehouse operation equipment target is in a motion state, it is further determined whether there is a person target in the preset range of the warehouse operation equipment target, and if so, an alarm processing is performed.

[0058] As can be seen from the above, when the target recognition is performed, the present embodiment method needs to meet the same category recognition result of the continuous multiple picture frames, and the difference between the time stamps corresponding to the adjacent two picture frames of the continuous multiple picture frames is less than the first threshold value, so as to determine the recognition result of the continuous multiple picture frames as an effective recognition result, which guarantees the timeliness and accuracy of the recognition result, and further judges whether the worker wears a safety helmet based on the effective recognition result. At the same time, the state recognition of the warehouse operation equipment is improved, the misrecognition problem caused by the identification frame jitter is effectively avoided, and the false alarm rate is greatly reduced.

[0059] The warehouse alarm device of one or more embodiments of the present application will be described in detail below. Those skilled in the art can understand that these alarm devices can all be configured using commercially available hardware components through the steps taught by the present solution. Figure 3 An embodiment of the present application provides a warehouse alarm device, as shown in Figure 3 The alarm device includes a video acquisition module 11, an identification module 12, a first alarm module 13, and a second alarm module 14.

[0060] The video acquisition module 11 is used to acquire the video in the warehouse in real time.

[0061] The recognition module 12 recognizes pictures of the video through an AI target detection model, obtains a current timestamp, and obtains a recognition result, the recognition result including a recognized category, recognized box coordinates (two opposite coordinates), and a corresponding timestamp, the recognized category including a person target, a person target wearing a safety helmet, a person target not wearing a safety helmet, and a warehouse operation equipment target, wherein for recognition of each category, if the category recognition results of continuous multiple pictures (such as 3 continuous pictures) are the same, and the difference between the timestamps corresponding to the adjacent two pictures of the continuous multiple pictures is less than a first threshold, then the recognition results of the continuous multiple pictures are all valid recognition results.

[0062] It should be noted that in the prior art, the recognition result of a single picture is directly used, which cannot guarantee the timeliness of the result, and when the AI target detection is misrecognized, the recognition result is inaccurate, and in the present application, the recognition result of continuous multiple pictures is only determined as a valid recognition result when the category recognition results of the continuous multiple pictures are the same and the difference between the timestamps corresponding to the adjacent two pictures of the continuous multiple pictures is less than a first threshold, thereby guaranteeing the timeliness and accuracy of the recognition result, for example, when the recognition result is ABBAAA, obviously, the difference between the timestamp of the second A and the timestamp of the first A will be greater than the first threshold, then ABB is an invalid recognition result, and similarly, AAA is a valid recognition result, wherein A represents wearing a safety helmet, and B represents not wearing a safety helmet.

[0063] The scene to which the method of the embodiment of the present application is applied usually has only one warehouse operation equipment, such as a forklift, and the person target usually has multiple, therefore, in order to distinguish different person targets and the corresponding recognition box coordinates and timestamps, the different person targets are further tracked according to the recognition result:

[0064] The recognition results of all person targets are input into a tracking model, and the ID of each person target is returned by the model, so that when the person target does not disappear, the ID of the person target will not change even if the position of the person target changes, such as person1, person2...personN.

[0065] Further, in order to correspond the person target wearing a safety helmet and the person target not wearing a safety helmet to different person targets, the recognition box of the person target wearing a safety helmet and the recognition box of the person target not wearing a safety helmet are determined to belong to the recognition box of the person target respectively through the recognition box coordinates, such as through calculation of the recognition box coordinates, the recognition box a of the person target wearing a safety helmet is in the recognition box A of person1, and then the recognition box a belongs to the recognition box A.

[0066] In the present embodiment, the recognition result is saved through a structure body:

[0067] First, a structure body is created, and the structure body has a member corresponding to each category.

[0068] Then, the identified bounding box coordinates of the same category and the corresponding time stamps obtained each time are saved in the corresponding member in turn, such as:

[0069] ‘person1’ : [{timestamp: bounding box coordinates}, {{timestamp: bounding box coordinates},...]

[0070] ‘person1_DM’ : [{timestamp: bounding box coordinates}, {{timestamp: bounding box coordinates},...]

[0071] ‘person1_WDM’ : [{timestamp: bounding box coordinates}, {{timestamp: bounding box coordinates},...]

[0072] ‘person2’ : [{timestamp: bounding box coordinates}, {{timestamp: bounding box coordinates},...] ...

[0073] ‘folklift’ : [{timestamp: bounding box coordinates}, {{timestamp: bounding box coordinates},...]

[0074] Among them, person1_DM represents that the person target 1 wears a safety helmet, person1_DM represents that the person target 1 does not wear a safety helmet, and folklift represents a forklift.

[0075] Based on the above structure, if the difference between the currently saved time stamp and the previous time stamp in the same structure member is greater than or equal to the first threshold, all bounding box coordinates and corresponding time stamps before the currently saved time stamp in the member are deleted, such as:

[0076] ‘person1_WDM’ : [{timestamp1: bounding box coordinates1}, {{timestamp2: bounding box coordinates2}, {{timestamp3: bounding box coordinates3}], if the difference between the currently saved time stamp 3 and the time stamp 2 is greater than or equal to the first threshold, {timestamp1: bounding box coordinates1}, {{timestamp2: bounding box coordinates2} are deleted, and when there are 3 time stamps in the ‘person1_WDM’ member, that is, ‘person1_WDM’ : [{timestamp3: bounding box coordinates3}, {{timestamp4: bounding box coordinates4}, {{timestamp5: bounding box coordinates5}], it represents that the person target 1 wears a safety helmet in the continuous 3 frames of pictures, and the difference between the time stamps corresponding to the adjacent two frames of pictures before and after the continuous 3 frames of pictures is less than the first threshold. It can be seen that after such processing, each member is equivalent to a set of effective recognition results of the corresponding category.

[0077] The first alarm module 13 is configured to determine the real-time position of the human target according to the human target bounding box coordinates of the valid recognition result, and if the real-time position is located in a preset dangerous area, further determine whether the human target wears a safety helmet, and if not, perform alarm processing.

[0078] The second alarm module 14 is configured to determine the state of the warehouse operation equipment target in real time: according to the valid recognition results of the front and rear adjacent two frames of pictures, determine whether the warehouse operation equipment target bounding box satisfies the following conditions, if all satisfy, the warehouse operation equipment target is in a motion state:

[0079] Condition 1: the center point offset of the warehouse operation equipment target bounding box of the front and rear adjacent two frames of pictures exceeds a second threshold value;

[0080] Condition 2: the coincidence degree of the warehouse operation equipment target bounding box of the front and rear adjacent two frames of pictures is less than a second threshold value;

[0081] Condition 3: there are two pairs of diagonal points between the warehouse operation equipment target bounding boxes of the front and rear adjacent two frames of pictures that satisfy the deviation condition, wherein the two points with the same position in the warehouse operation equipment target bounding boxes of the front and rear adjacent two frames of pictures constitute a pair of diagonal points, and the deviation condition is satisfied, that is, the absolute value of the x coordinate difference or the absolute value of the y coordinate difference of the two corners in a pair of diagonal points is greater than or equal to a third threshold value.

[0082] In the prior art, the identification of the target static and motion is usually identified only by using condition 1 and condition 2. When the warehouse operation equipment is working, the operator (such as a forklift driver) on the equipment is exposed in the video picture with a high proportion, so when the equipment is identified, the AI model will probably identify the object or person close to the equipment as part of the equipment, which leads to the size of the identification box of the equipment in the front and rear adjacent picture frames to change (the box to shake) even if the equipment is parked and not working. Once the identification box changes greatly, only using condition 1 and condition 2 will misidentify the equipment as a motion state, and the present application increases condition 3 to effectively avoid misidentification, such as Figure 2 as shown in Figure 2 (a), the upper left corner points of the two identification boxes coincide (the coordinate difference is 0, which is less than the third threshold value), and the lower left corner points coincide (the coordinate difference is 0, which is less than the third threshold value), and the identification is in a static state, Figure 2 (b), the four corner points of the two identification boxes are far away (the coordinate difference is greater than or equal to the third threshold value), and the identification is in a motion state, Figure 2 (c), the upper left corner points of the two identification boxes coincide (the coordinate difference is 0, which is less than the third threshold value), and the lower left corner points are close (the coordinate difference is greater than 0, which is less than the third threshold value), and the identification is in a static state.

[0083] If the warehouse operation equipment target is in a moving state, it is further determined whether there is a human target in the preset range of the warehouse operation equipment target, and if yes, an alarm is processed.

[0084] In summary, the warehouse alarm device provided by the above embodiments can execute the warehouse alarm method provided by each of the above embodiments.

[0085] As the same as the above idea, the warehouse alarm device provided by the above embodiments can execute the warehouse alarm method provided by each of the above embodiments. Figure 3 As shown in the warehouse alarm device provided by the above embodiments can be implemented as an electronic device, Figure 4 As shown in the warehouse alarm device provided by the above embodiments can be implemented as an electronic device,

[0086] For example, the electronic device includes a storage module 21 and a processor 22, the storage module 21 includes instructions loaded and executed by the processor 22, and the instructions make the processor 22 execute the steps of the warehouse alarm method according to various exemplary embodiments of the present application described in the above part of the specification when executed.

[0087] It should be understood that the processor 22 can be a central processing unit (CPU), and the processor 22 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0088] In the embodiments of the present application, a computer readable storage medium is also provided, the computer readable storage medium stores one or more programs, and the one or more programs implement the steps of the warehouse alarm method according to various exemplary embodiments of the present application described in the above part when executed by the processor.

[0089] Those of ordinary skill in the art will understand that all or some of the steps, functions, modules / units in the methods disclosed above and the functions of the modules / units in the systems and devices can be implemented as software, firmware, hardware, or any appropriate combination thereof. In hardware implementation, the division between the functional modules / units referred to in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components working in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable storage media, which can include computer-readable storage media (or non-transitory media) and communication media (or transitory media).

[0090] As is well known to those of ordinary skill in the art, the term computer readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, it is well known to those of ordinary skill in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any information delivery media.

[0091] By way of example, the computer readable storage media can be an internal storage unit of the electronic device of the aforementioned embodiments, such as a hard disk or a memory of the electronic device. The computer readable storage media can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, or the like.

[0092] The electronic device and the computer readable storage medium provided by the foregoing embodiments require that, when target recognition is performed, the category recognition results of continuous multiple frames of pictures are the same, and the difference between the time stamps corresponding to the adjacent two frames of pictures of the continuous multiple frames of pictures is less than the first threshold, so that the recognition result of the continuous multiple frames of pictures is determined as a valid recognition result, the timeliness and accuracy of the recognition result are ensured, and then it is judged whether the worker wears a safety helmet based on the valid recognition result. Meanwhile, the state recognition of the warehouse operation equipment is improved, the misrecognition problem caused by the shaking of the recognition box is effectively avoided, and the false alarm rate is greatly reduced.

[0093] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and the equivalent technologies thereof, the present application also intends to include these modifications and variations.

Claims

1. A warehouse alarm method, characterized in that, include: Real-time acquisition of video from within the warehouse; The images in the video are identified, and the current timestamp is obtained to obtain the identification result. The identification result includes the identified category, the coordinates of the identification box, and the corresponding timestamp. The identified categories include human targets, human targets wearing safety helmets, human targets not wearing safety helmets, and warehouse operation equipment targets. For each category, if the category identification results of multiple consecutive frames are the same, and the difference between the timestamps of two consecutive frames is less than a first threshold, then the identification results of the multiple consecutive frames are all valid identification results. Based on the coordinates of the human target recognition box of the effective recognition result, the real-time position of the human target is determined. If the real-time position is located in a preset danger zone, it is further determined whether the human target is wearing a safety helmet. If not, an alarm is triggered. Real-time determination of the state of the warehouse operation equipment target: Based on the valid recognition results of two consecutive frames of images, determine whether the warehouse operation equipment target recognition box meets the following conditions. If all conditions are met, the warehouse operation equipment target is in motion: The offset of the center point of the target recognition box of warehouse operation equipment in two consecutive frames exceeds the second threshold. The overlap of the target recognition boxes of warehouse operation equipment in two consecutive frames is less than the second threshold. There are two pairs of corner points in the target recognition boxes of warehouse operation equipment in two consecutive frames of images that meet the deviation condition. Specifically, two points in the same position in the target recognition boxes of warehouse operation equipment in two consecutive frames of images form a pair of corner points. Meeting the deviation condition means that the absolute value of the difference between the x coordinates or the absolute value of the difference between the y coordinates of the two corner points is greater than or equal to the third threshold. If the target of the warehouse operation equipment is in motion, it is further determined whether there is a human target within the preset range of the target of the warehouse operation equipment. If so, an alarm is triggered.

2. The warehouse alarm method according to claim 1, characterized in that, Also includes: Based on the recognition results, target tracking is performed on different human targets to distinguish different human targets and their corresponding recognition box coordinates and timestamps; The identification boxes for the person wearing a helmet and the person not wearing a helmet are determined by the identification box coordinates, so as to associate the person wearing a helmet and the person not wearing a helmet with different person targets.

3. The warehouse alarm method according to claim 2, characterized in that, The step of recognizing images from the video, obtaining the current timestamp, and getting the recognition result further includes: Create a structure that has members that correspond one-to-one with the category; The coordinates of the recognition boxes of the same category obtained each time and the corresponding timestamp are saved sequentially in the corresponding member.

4. A warehouse alarm method according to claim 3, characterized in that, Also includes: If the difference between the currently saved timestamp and the previous timestamp in the same structure member is greater than or equal to the first threshold, then delete all the identification box coordinates and corresponding timestamps before the currently saved timestamp in the member.

5. A warehouse alarm method according to claim 4, characterized in that, The warehouse operation equipment is a forklift.

6. A warehouse alarm method according to claim 5, characterized in that, The consecutive multi-frame images refer to three consecutive frames.

7. A warehouse alarm device, characterized in that, include: The video acquisition module is used to acquire videos from the warehouse in real time. The recognition module is used to recognize the images in the video, obtain the current timestamp, and obtain the recognition result. The recognition result includes the recognized category, the coordinates of the recognition box, and the corresponding timestamp. The recognized categories include human targets, human targets wearing safety helmets, human targets not wearing safety helmets, and warehouse operation equipment targets. For each category, if the category recognition results of multiple consecutive frames are the same, and the difference between the timestamps of two consecutive frames is less than a first threshold, then the recognition results of the multiple consecutive frames are all valid recognition results. The first alarm module is used to determine the real-time location of the human target based on the coordinates of the human target recognition box of the effective recognition result. If the real-time location is located in a preset danger zone, it is further determined whether the human target is wearing a safety helmet. If not, an alarm is triggered. The second alarm module is used to determine the status of the warehouse operation equipment target in real time: based on the valid recognition results of two consecutive frames of images, it determines whether the warehouse operation equipment target recognition box meets the following conditions. If all conditions are met, the warehouse operation equipment target is in motion: The offset of the center point of the target recognition box of warehouse operation equipment in two consecutive frames exceeds the second threshold. The overlap of the target recognition boxes of warehouse operation equipment in two consecutive frames is less than the second threshold. There are two pairs of corner points in the target recognition boxes of warehouse operation equipment in two consecutive frames of images that meet the deviation condition. Specifically, two points in the same position in the target recognition boxes of warehouse operation equipment in two consecutive frames of images form a pair of corner points. Meeting the deviation condition means that the absolute value of the difference between the x coordinates or the absolute value of the difference between the y coordinates of the two corner points is greater than or equal to the third threshold. If the target of the warehouse operation equipment is in motion, it is further determined whether there is a human target within the preset range of the target of the warehouse operation equipment. If so, an alarm is triggered.

8. An electronic device, characterized in that, The system includes a storage module comprising instructions loaded and executed by a processor, the instructions, when executed, causing the processor to perform a warehouse alarm method according to any one of claims 1-6.

9. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by the processor, they implement a warehouse alarm method according to any one of claims 1-6.

Images