Logistics safety monitoring system based on AI video identification technology and working method thereof

Abstract

The invention discloses a logistics safety monitoring system based on an AI video recognition technology and a working method thereof, and relates to the technical field of intelligent logistics. The logistics safety monitoring system comprises a multi-source video acquisition module, an AI intelligent analysis engine, a logistics safety decision center and a cooperative control interface which are in communication connection in sequence, and the multi-source video acquisition module is used for acquiring video stream data of an article handover process between an unmanned aerial vehicle and a rider in real time from multiple angles; the AI intelligent analysis engine is used for carrying out multi-modal identification analysis on the video stream data in real time based on an AI video identification technology to obtain a multi-modal identification result, and the logistics safety decision center is used for generating a logistics safety decision in real time according to the identification result. And the cooperative control interface is used for converting the safety decision into a control instruction in real time and distributing the control instruction to the unmanned aerial vehicle and / or the rider terminal, so that the safety risk of cooperation of the unmanned aerial vehicle and the rider can be effectively monitored, and a corresponding decision is made to realize comprehensive guarantee of logistics safety.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent logistics, and particularly relates to a logistics safety monitoring system based on AI video recognition technology and a working method thereof. BACKGROUND

[0002] The "last mile" of logistics distribution is one of the most complex and costliest links in modern supply chains. In recent years, unmanned aerial vehicle (UAV) distribution technology has been developing rapidly, but due to regulatory restrictions and technical maturity, fully autonomous UAV distribution solutions still face many challenges. In particular, in densely populated areas, UAVs face difficulties such as safe landing and precise delivery. To this end, some people have proposed a distribution mode in which UAVs and riders collaborate, with riders responsible for the final delivery to customers or acting as intermediaries between UAVs and customers. However, this collaborative mode introduces new safety risks, particularly during the transfer of goods, which may result in damage, misplacement, loss of goods, and unauthorized personnel accessing goods. Existing logistics monitoring systems are mostly targeted at traditional warehouse or transportation links, and lack a safety monitoring solution specifically for the UAV and rider collaboration scenario. Therefore, there is an urgent need for a specialized technical solution that can comprehensively monitor the safety of the transfer of goods in this innovative collaboration mode based on AI video recognition technology. SUMMARY

[0003] The purpose of the present application is to provide a logistics safety monitoring system based on AI video recognition technology and a working method thereof, to solve the problem that existing logistics monitoring solutions cannot monitor the safety risks of UAV and rider collaboration.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: In a first aspect, a logistics safety monitoring system based on AI video recognition technology is provided, comprising a multi-source video acquisition module, an AI intelligent analysis engine, a logistics safety decision center, and a collaborative control interface, which are connected in sequence. The multi-source video acquisition module is configured to acquire video stream data of the transfer of goods between a UAV and a rider in real time from multiple angles, including the UAV side angle and the rider side angle, and transmit the video stream data to the AI intelligent analysis engine in real time. The AI intelligent analysis engine is configured to perform multi-modal recognition analysis on the video stream data in real time based on AI video recognition technology, obtain multi-modal recognition results, and transmit the multi-modal recognition results to the logistics safety decision center in real time, wherein the multi-modal recognition results include transfer goods recognition results, rider behavior recognition results, human-machine legality recognition results, and transfer environment recognition results. The logistics security decision center is configured to generate a logistics security decision in real time according to the multi-modal recognition result in combination with a preset article handover security rule and / or a multi-factor risk assessment model, and transmit the logistics security decision to the collaborative control interface in real time, wherein the logistics security decision includes any one of prohibition of handover, normal handover, suspension of handover, and risk warning. The collaborative control interface is configured to convert the logistics security decision into a control instruction in real time, and transmit the control instruction to the UAV via a UAV communication link and / or to a rider terminal via a rider terminal communication link.

[0005] Based on the above invention content, a new logistics security monitoring solution designed for a UAV and rider collaboration scenario is provided, which includes a multi-source video acquisition module, an AI intelligent analysis engine, a logistics security decision center, and a collaborative control interface connected in sequence, wherein the multi-source video acquisition module is configured to acquire video stream data of an article handover process between a UAV and a rider from multiple angles in real time, the AI intelligent analysis engine is configured to perform multi-modal recognition analysis on the video stream data in real time based on AI video recognition technology to obtain a multi-modal recognition result, the logistics security decision center is configured to generate a logistics security decision in real time according to the multi-modal recognition result, and the collaborative control interface is configured to convert the logistics security decision into a control instruction in real time and distribute it to the UAV and / or the rider terminal. Thus, through video intelligent recognition and analysis of the transferred article, the rider, and related environmental elements, the safety risks of the UAV and rider collaboration can be effectively monitored, and corresponding decisions can be made to achieve comprehensive protection of logistics security, facilitating practical application and promotion.

[0006] In one possible design, the multi-source video acquisition module includes an on-board camera unit configured to acquire video stream data of the article handover process from a UAV side in real time, and a rider personal camera unit configured to acquire video stream data of the article handover process from a rider side in real time, wherein the on-board camera unit includes a panoramic camera installed on the top of the UAV, a zoom camera installed on the bottom of the UAV, and / or an infrared camera installed on the UAV, and the rider personal camera unit includes a first camera installed on a helmet of the rider, a second camera installed on a delivery tool of the rider, and / or a third camera located on a rider terminal.

[0007] In one possible design, based on AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain a multi-modal recognition result, including: According to the video stream data, a handover article detection model pre-trained based on a YOLO target detection algorithm is used to detect the packaging integrity information of the handover article in real time, wherein the packaging integrity information includes a packaging abnormal state based on the detection of the outer packaging video image of the handover article and used to indicate whether the outer packaging is damaged, deformed and / or opened; According to the video stream data, a Deepsort algorithm is used to track the handover article in real time to obtain a real-time motion trajectory of the handover article, and whether there is an article throwing situation or a natural falling situation is determined in real time according to the real-time motion trajectory, to obtain abnormal motion detection information of the handover article. The packaging integrity information and the abnormal motion detection information are summarized in real time to obtain a handover article recognition result.

[0008] In one possible design, based on AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain multi-modal recognition results, including: According to the video stream data, a rider detection model pre-trained based on a YOLO target detection algorithm and a human action recognition model pre-trained based on a human skeleton key point detection algorithm are used to identify the action behavior of the rider in real time, and the action behavior is matched to a normal behavior or an abnormal behavior in real time by combining a preset normal behavior library and / or an abnormal behavior library, to obtain behavior state detection information. According to the video stream data, a Deepsort algorithm is used to track the rider in real time to obtain a real-time motion trajectory of the rider, and whether the change timing of the relative position and / or distance between the unmanned aerial vehicle and the rider conforms to a preset article handover safety specification is analyzed in real time according to the real-time motion trajectory, to obtain human-machine interaction detection information. The behavior state detection information and the human-machine interaction detection information are summarized in real time to obtain a rider behavior recognition result.

[0009] In one possible design, based on AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain multi-modal recognition results, including: According to the video stream data, a gait feature recognition algorithm, a clothing feature recognition algorithm and a delivery tool feature recognition algorithm are used to identify the gait feature, the clothing feature and the delivery tool feature of the rider one by one in real time, and the identity legality of the rider is verified according to the gait feature, the clothing feature and the delivery tool feature, to obtain rider legality verification information. According to the video stream data, a UAV visual feature recognition algorithm is adopted to recognize the visual feature of the UAV in real time, and the handover task matching of the UAV and the rider is verified according to the visual feature, the gait feature, the clothing feature and the delivery tool feature of the rider, to obtain task matching verification information; The rider legality verification information and the task matching verification information are summarized in real time to obtain a man-machine legality recognition result.

[0010] In one possible design, based on AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain a multi-modal recognition result, including: According to the video stream data, an obstacle detection model pre-trained based on a YOLO target detection algorithm is adopted to detect obstacles located around the UAV in real time, and whether a safe place suitable for article handover exists directly below the UAV is analyzed in real time according to the obstacle detection result, to obtain safe place detection information; According to the video stream data, a rider detection model and a personnel detection model pre-trained based on a YOLO target detection algorithm respectively are adopted to detect intruding personnel located directly below the UAV in real time, and the density, the direction of movement and / or the distance to the UAV of the intruding personnel are analyzed in real time according to the intruding personnel detection result, to obtain personnel intrusion detection information, wherein the intruding personnel refers to personnel other than the rider; According to the video stream data, environmental condition detection information of the surrounding environment of the UAV is analyzed in real time, wherein the environmental condition detection information includes weather conditions and / or illumination conditions; The safe place detection information, the personnel intrusion detection information and the environmental condition detection information are summarized in real time to obtain a handover environment recognition result.

[0011] In one possible design, according to the multi-modal recognition result, a logistics safety decision is generated in real time in combination with a preset article handover safety rule, including: According to the multi-modal recognition result and the preset article handover safety rule, if it is found in real time that no safe place suitable for article handover exists directly below the UAV and / or the surrounding environment of the UAV is in a harsh condition, a logistics safety decision for indicating prohibition of handover is generated in real time; And / or, according to the multi-modal recognition result and the preset article handover safety rule, if it is found in real time that a safe place suitable for article handover exists directly below the UAV, the surrounding environment of the UAV is in a normal condition, the identity legality verification of the rider is passed, and the handover task matching verification of the UAV and the rider is passed, a logistics safety decision for indicating normal handover is generated in real time; And / or, according to the multi-modal recognition result and the preset article handover safety rule, if it is found in real time that the safe place directly below the UAV becomes a non-safe place, the surrounding environment of the UAV becomes a bad situation, the article to be handed over appears an incomplete package situation / abnormal motion situation, the rider makes an abnormal behavior, or an irregular human-machine interaction situation appears between the UAV and the rider, a logistics safety decision for indicating suspension of handover is generated in real time; And / or, according to the multi-modal recognition result and the preset article handover safety rule, if it is found in real time that there is a personnel intrusion situation directly below the UAV, a logistics safety decision for indicating risk warning is generated in real time.

[0012] In one possible design, according to the multi-modal recognition result, a logistics safety decision is generated in real time in combination with a preset article handover safety rule and a multi-factor risk assessment model, including: According to the multi-modal recognition result and the preset article handover safety rule, an initial logistics safety decision is generated in real time, wherein the logistics safety decision includes any one of prohibition of handover, normal handover, suspension of handover, and risk warning; If the initial logistics safety decision does not indicate prohibition of handover, according to the multi-modal recognition result and a preset multi-factor risk assessment model, four factor risk assessment scores respectively corresponding to the article handover recognition result, the rider behavior recognition result, the human-machine legality recognition result, and the handover environment recognition result are respectively evaluated in real time, and a risk assessment total score is calculated by using a weighted average method according to the four factor risk assessment scores; According to the risk assessment total score and a decision switching matrix containing MxN elements in the article handover safety rule, it is determined whether the initial logistics safety decision needs to be adjusted, if yes, a final logistics safety decision is obtained by adjustment, otherwise the initial logistics safety decision is directly taken as the final logistics safety decision, wherein M and N respectively represent positive integers, each row index of the decision switching matrix respectively corresponds to a logistics safety decision not indicating prohibition of handover, each column index of the decision switching matrix respectively corresponds to an evaluation score interval, and an element value of the element is a unique number of the logistics safety decision.

[0013] In one possible design, the system further includes a data recording platform in communication connection with the multi-source video acquisition module, the AI intelligent analysis engine, and the logistics safety decision center respectively; The data recording platform is configured to collect and store the video stream data, the multi-modal recognition result, and the logistics safety decision.

[0014] In a second aspect, a working method of the logistics security monitoring system based on the AI video recognition technology according to the first aspect or any possible design of the first aspect is provided, comprising: collecting, by the multi-source video acquisition module, video stream data of the article handover process between the UAV and the rider in real time from multiple angles, wherein the multiple angles include a UAV side angle and a rider side angle; performing, by the AI intelligent analysis engine, multi-modal recognition analysis on the video stream data in real time based on the AI video recognition technology to obtain multi-modal recognition results, wherein the multi-modal recognition results include article handover recognition results, rider behavior recognition results, man-machine legality recognition results, and handover environment recognition results; generating, by the logistics security decision center, a logistics security decision in real time according to the multi-modal recognition results in combination with a preset article handover safety rule and a multi-factor risk assessment model, wherein the logistics security decision includes any one of prohibition of handover, normal handover, suspension of handover, and risk warning; converting, by the cooperative control interface, the logistics security decision into a control instruction in real time, and transmitting the control instruction to the UAV in real time through a UAV communication link and / or to a rider terminal in real time through a rider terminal communication link.

[0015] The above-mentioned scheme has the following beneficial effects: (1) The present application provides a new logistics security monitoring scheme designed for a UAV and rider cooperation scenario, i.e., comprising a multi-source video acquisition module, an AI intelligent analysis engine, a logistics security decision center, and a cooperative control interface connected in sequence, wherein the multi-source video acquisition module is used to collect video stream data of the article handover process between the UAV and the rider in real time from multiple angles, the AI intelligent analysis engine is used to perform multi-modal recognition analysis on the video stream data in real time based on the AI video recognition technology to obtain multi-modal recognition results, the logistics security decision center is used to generate a logistics security decision in real time according to the multi-modal recognition results, and the cooperative control interface is used to convert the logistics security decision into a control instruction in real time and distribute it to the UAV and / or the rider terminal, so that through video intelligent recognition and analysis of the transferred article, the rider, and related environmental elements, the safety risk of the UAV and rider cooperation can be effectively monitored, and corresponding decisions can be made to realize all-round protection of logistics security, facilitating practical application and promotion. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0017] Figure 1 The structural schematic diagram of the logistics safety monitoring system based on the AI video recognition technology provided by the embodiments of the present application.

[0018] Figure 2 The flowchart of the working method of the logistics safety monitoring system based on the AI video recognition technology provided by the embodiments of the present application. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor. It should be noted that the description of these embodiment modes is used to help understand the present application, but does not constitute a limitation on the present application.

[0020] It should be understood that although the terms first and second, etc. can be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, the first object can be called the second object, and similarly, the second object can be called the first object, without departing from the scope of the example embodiments of the present application.

[0021] It should be understood that for the term "and / or" which can appear in the present application, it only describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can mean that A exists alone, B exists alone or A and B exist simultaneously, etc. For example, A, B and / or C can mean that any one of A, B and C exists or any combination thereof. For the term " / and" which can appear in the present application, it describes another association object relationship, which means that there can be two kinds of relationships, for example, A / and B can mean that A exists alone or A and B exist simultaneously. In addition, for the character " / " which can appear in the present application, it generally means that the associated objects before and after are in an "or" relationship.

[0022] Embodiment one As Figure 1As shown, the logistics security monitoring system based on the AI video recognition technology provided by the embodiment includes but is not limited to a multi-source video acquisition module, an AI intelligent analysis engine, a logistics security decision center and a collaborative control interface which are sequentially connected in communication.

[0023] The multi-source video acquisition module is configured to acquire video stream data of an article handover process between a UAV and a rider from multiple angles in real time, and transmit the video stream data to the AI intelligent analysis engine in real time, wherein the multiple angles include but are not limited to a UAV side angle and a rider side angle, etc. The video stream data is used to record the whole article handover process from the UAV and the rider reaching a handover point to leaving the handover point respectively. In order to ensure the comprehensiveness of the video stream data, preferably, the multi-source video acquisition module includes but is not limited to an on-board camera unit for acquiring video stream data of the article handover process from the UAV side angle in real time, a rider personal camera unit for acquiring video stream data of the article handover process from the rider side angle in real time, etc., wherein the on-board camera unit includes but is not limited to a panoramic camera installed on the top of the UAV, a zoom camera installed on the bottom of the UAV and / or an infrared camera installed on the UAV, etc., and the rider personal camera unit includes but is not limited to a first camera installed on the helmet of the rider, a second camera installed on the delivery tool of the rider and / or a third camera located on the rider terminal, etc. The panoramic camera is used to shoot a panoramic view of the handover environment through a wide-angle lens to monitor surrounding personnel, vehicles and other potential interference objects, and specifically can be implemented by using a camera with the following parameters: 1080P resolution and 120-degree wide angle. The zoom camera has optical zoom and image stabilization functions, and can be used to capture close-up pictures of the article handover process, and specifically can be implemented by using a camera with the following parameters: 4K resolution and 20x hybrid zoom. The infrared camera is used for low-light monitoring to ensure all-weather monitoring capability. The first camera can be specifically located on the front of the helmet, and is used to capture the UAV and the handover article in the rider's field of view from a first-person perspective, and specifically can be implemented by using a camera with the following parameters: 1080P resolution and electronic image stabilization. The second camera can be specifically located on the handle or front box of the vehicle, and is used to shoot the interaction process between the rider and the UAV from a side view, and specifically can be implemented by using a camera with the following parameters: 720P resolution and waterproof design. The rider terminal can be specifically a smart phone, and is used to assist in acquiring video images from specific angles.

[0024] The AI intelligent analysis engine is configured to perform multi-modal recognition analysis on the video stream data in real time based on an AI video recognition technology, to obtain multi-modal recognition results, and to transmit the multi-modal recognition results to the logistics security decision center in real time, wherein the multi-modal recognition results include but are not limited to handover article recognition results, rider behavior recognition results, man-machine legality recognition results, and handover environment recognition results. The AI (Artificial Intelligence) intelligent analysis engine is one of the core processing devices of the entire system, and can preferably adopt an edge and cloud collaborative computing architecture, so that simple recognition tasks are processed by edge devices, and complex analysis tasks are uploaded to the cloud. The AI video recognition technology refers to a technology for information recognition and extraction based on video data through artificial intelligence technologies such as machine vision, for example, target detection technology, multi-target tracking technology, human skeleton key point detection technology, feature recognition technology, and environment detection technology; the present embodiment does not involve improving and innovating the existing AI video recognition technology, but focuses on modifying and applying the existing AI video recognition technology.

[0025] Specifically, based on the AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain multi-modal recognition results, including but not limited to the following steps S211-S213.

[0026] S211. According to the video stream data, a handover article detection model pre-trained based on a YOLO target detection algorithm is used to detect the packaging integrity information of the handover article in real time, wherein the packaging integrity information includes but is not limited to packaging abnormal states such as based on the outer packaging video image detection of the handover article and used to indicate whether the outer packaging has conditions such as damage, deformation, and / or opening, etc.

[0027] In the step S211, the YOLO (You Only Look Once) target detection algorithm is an existing target detection algorithm, the core idea of which is to convert the target detection task into a regression problem, to directly predict the position and category of the target through a single neural network, to realize an end-to-end detection process, and thus the handover article detection model and subsequent rider detection model, obstacle detection model, and personnel detection model, etc. can be pre-trained in combination with a conventional rate setting modeling process. In addition, the number and type of the handover article can also be detected and added to the handover article recognition results, to achieve the purpose of rich recognition results.

[0028] S212. According to the video stream data, a Deepsort algorithm is used to track the target of the handover article in real time to obtain a real-time motion trajectory of the handover article, and whether there is an article throwing situation or a natural falling situation is judged in real time according to the real-time motion trajectory, and abnormal motion detection information of the handover article is obtained.

[0029] In the step S212, the Deepsort (Deep Simple Online and Realtime Tracking) algorithm is a classic algorithm in the field of multi-target tracking, which can introduce deep learning features on the basis of the SORT (Simple Online and Realtime Tracking) algorithm to improve the tracking accuracy, so as to achieve the purpose of tracking the target of the handover article to obtain the real-time motion trajectory.

[0030] S213. Real-time summary of the package integrity information and the abnormal motion detection information, and obtaining the handover article identification result.

[0031] Specifically, based on AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain multi-modal recognition results, including but not limited to the following steps S221-S223.

[0032] S221. According to the video stream data, a rider detection model based on a YOLO target detection algorithm and a human action recognition model based on a human skeleton key point detection algorithm are used to identify the action behavior of the rider in real time, and the action behavior is matched to normal behavior or abnormal behavior in real time by combining a preset normal behavior library and / or an abnormal behavior library, and behavior state detection information is obtained.

[0033] In the step S221, the human skeleton key point detection algorithm is an existing algorithm that automatically identifies and locates key skeleton points (such as joints, facial features, etc.) in human images or videos through computer vision technology, thereby describing the human posture, so the human action recognition model can be pre-trained by combining a conventional rating modeling process, and after the rider video stream data is obtained from the video stream data based on the rider detection result obtained by the rider detection model, the human action recognition model is applied to the rider video stream data for human action recognition processing. The normal behavior library is used to store normal behaviors such as normal delivery, and the abnormal behavior library is used to store abnormal behaviors such as rough handling and / or hiding articles, so that the action behavior of the rider can be matched and classified by combining existing matching algorithms, and the matching result is used as the behavior state detection information.

[0034] S222. According to the video stream data, a Deepsort algorithm is used to track the rider in real time to obtain a real-time motion trajectory of the rider, and whether a change timing of a relative position and / or distance between the UAV and the rider conforms to a preset item handover safety specification is analyzed in real time according to the real-time motion trajectory, to obtain human-machine interaction detection information.

[0035] In the step S222, the item handover safety specification may be, for example, “whether the UAV is hovering at a safe height (usually 1-2 meters higher than the head of the rider)”, “whether the rider forcibly takes the item in an unstable state of the UAV”, and the like.

[0036] S223. The behavior state detection information and the human-machine interaction detection information are summarized in real time to obtain a rider behavior recognition result.

[0037] Specifically, based on an AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain a multi-modal recognition result, including but not limited to the following steps S231-S233.

[0038] S231. According to the video stream data, a gait feature recognition algorithm, a clothing feature recognition algorithm, and a delivery tool feature recognition algorithm are used to correspondingly identify the gait feature, the clothing feature, and the delivery tool feature of the rider in real time, and the identity legitimacy of the rider is verified according to the gait feature, the clothing feature, and the delivery tool feature, to obtain rider legitimacy verification information.

[0039] In the step S231, the gait feature recognition algorithm is used to extract the posture feature of a human body when walking for identity recognition, the clothing feature recognition algorithm is used to extract the identification feature (such as a badge and a company logo, etc.) of the clothing of the rider, and the delivery tool feature recognition algorithm is used to extract the specific feature (such as a model and a color of an electric vehicle, etc.) of the delivery tool of the rider, and these recognition algorithms can be realized by referring to conventional feature extraction technology with routine modification. In addition, the specific verification process of the identity legitimacy of the rider can be realized by using a conventional feature matching algorithm.

[0040] S232. According to the video stream data, a UAV visual feature recognition algorithm is used to identify the visual feature of the UAV in real time, and the handover task matching of the UAV and the rider is verified according to the visual feature and the gait feature, the clothing feature, and the delivery tool feature of the rider, to obtain task matching verification information.

[0041] In the step S232, the UAV visual feature recognition algorithm is used to extract the visual features of the UAV (such as UAV body code and two-dimensional code, etc.), which can also be realized by modifying the existing feature extraction technology. In addition, the specific verification process of the matching of the UAV and the rider can also be realized by using the conventional feature matching algorithm.

[0042] S233. Real-time aggregation of the rider legal verification information and the task matching verification information to obtain a human-computer legal recognition result.

[0043] Specifically, based on the AI video recognition technology, the video stream data is analyzed in real time to obtain a multi-modal recognition result, including but not limited to the following steps S241-S244.

[0044] S241. According to the video stream data, an obstacle detection model based on a pre-trained YOLO target detection algorithm is used to detect obstacles around the UAV in real time, and whether there is a safe place suitable for goods handover under the UAV is analyzed in real time according to the obstacle detection result to obtain safe place detection information.

[0045] In the step S241, for example, when only one obstacle, flat ground, is found under the UAV according to the obstacle detection result, it is determined that there is a safe place suitable for goods handover.

[0046] S242. According to the video stream data, a rider detection model and a personnel detection model based on a pre-trained YOLO target detection algorithm are used to detect intruding personnel under the UAV in real time, and the density, direction of travel and / or distance to the UAV of the intruding personnel are analyzed in real time according to the intruding personnel detection result to obtain personnel intrusion detection information, wherein the intruding personnel refers to other personnel except the rider.

[0047] In the step S242, the complement of the rider detection result and the personnel detection result under the UAV can be used as the intruding personnel detection result.

[0048] S243. According to the video stream data, the environmental condition detection information of the surrounding environment of the UAV is analyzed in real time, wherein the environmental condition detection information includes weather conditions and / or lighting conditions.

[0049] In the step S243, the specific analysis process of the above-mentioned environmental conditions can be realized by modifying the existing analysis technology, for example, combining target detection algorithm and action recognition algorithm to detect the swing amplitude of branches to determine whether there is a strong wind weather condition, etc.

[0050] S244. Real-time aggregate the security site detection information, the personnel intrusion detection information and the environmental condition detection information to obtain a handover environment identification result.

[0051] The logistics security decision center is configured to generate a logistics security decision in real time according to the multimodal identification result and in combination with a preset article handover security rule and a multi-factor risk assessment model, and transmit the logistics security decision to the collaborative control interface in real time. The logistics security decision includes any one of, for example, a prohibition of handover, a normal handover, a suspension of handover, and a risk warning. Specifically, the logistics security decision generated in real time according to the multimodal identification result and in combination with a preset article handover security rule includes, but is not limited to, the following: according to the multimodal identification result and the preset article handover security rule, if it is found in real time that there is no security site suitable for article handover under the drone and / or the surrounding environment of the drone is in a poor condition, a logistics security decision for indicating a prohibition of handover is generated in real time; and / or, according to the multimodal identification result and the preset article handover security rule, if it is found in real time that there is a security site suitable for article handover under the drone, the surrounding environment of the drone is in a normal condition, the identity legality verification of the rider is passed, and the matching verification of the handover task between the drone and the rider is passed, a logistics security decision for indicating a normal handover is generated in real time; and / or, according to the multimodal identification result and the preset article handover security rule, if it is found in real time that the security site under the drone becomes a non-security site, the surrounding environment of the drone becomes a poor condition, the handover article appears an incomplete packaging condition / abnormal motion condition, the rider makes an abnormal behavior, or an abnormal human-machine interaction occurs between the drone and the rider, a logistics security decision for indicating a suspension of handover is generated in real time; and / or, according to the multimodal identification result and the preset article handover security rule, if it is found in real time that there is a personnel intrusion condition under the drone, a logistics security decision for indicating a risk warning is generated in real time; and the like. In addition, in order to achieve the purpose of fine-tuning the logistics security decision based on the risk assessment result, preferably, the logistics security decision is generated in real time according to the multimodal identification result, in combination with a preset article handover security rule and a multi-factor risk assessment model, and includes, but is not limited to, the following steps S301-S303.

[0052] S301. According to the multimodal identification result and the preset article handover security rule, an initial logistics security decision is generated in real time. The logistics security decision includes any one of, for example, a prohibition of handover, a normal handover, a suspension of handover, and a risk warning.

[0053] In the step S301, the specific generation details of the initial logistics security decision can refer to the details of generating the logistics security decision above, which will not be repeated here.

[0054] S302. If the initial logistics security decision does not indicate to prohibit the delivery, according to the multi-modal recognition result and the preset multi-factor risk assessment model, four factor risk assessment scores corresponding to the delivery article recognition result, the rider behavior recognition result, the man-machine legality recognition result and the delivery environment recognition result are respectively evaluated in real time, and a risk assessment total score is calculated by using a weighted average method according to the four factor risk assessment scores.

[0055] In the step S302, for example, when the initial logistics security decision indicates normal delivery, suspension of delivery or risk warning, the multi-factor risk assessment model can be enabled to obtain the risk assessment total score.

[0056] S303. According to the risk assessment total score and a decision switching matrix containing MxN elements in the delivery security rule, it is judged whether the initial logistics security decision needs to be adjusted. If yes, the final logistics security decision is obtained by adjustment, otherwise the initial logistics security decision is directly taken as the final logistics security decision, wherein M and N respectively represent positive integers, each row number of the decision switching matrix corresponds to a logistics security decision which does not indicate to prohibit the delivery, each column number of the decision switching matrix corresponds to an evaluation score interval, and the element value of the element is the unique number of the logistics security decision.

[0057] In the step S303, for example, the decision switching matrix can be shown in Table 1 as follows: Table 1. Example table of decision switching matrix (risk assessment score uses percentage system, M=3, N=3)

[0058] In the above table 1, "01" is the unique number of the logistics security decision for indicating normal delivery, "10" is the unique number of the logistics security decision for indicating risk warning, and "11" is the unique number of the logistics security decision for indicating suspension of delivery. Therefore, when the initial logistics security decision indicates risk warning, if the risk assessment total score is 20, the final logistics security decision for indicating normal delivery (i.e. "01") can be obtained by adjustment, if the risk assessment total score is 60, the logistics security decision remains unchanged, and if the risk assessment total score is 90, the final logistics security decision for indicating suspension of delivery (i.e. "11") can be obtained by adjustment.

[0059] The cooperative control interface is configured to convert the logistics safety decision into control instructions in real time, and transmit the control instructions to the UAV via a UAV communication link and / or transmit the control instructions to a rider terminal via a rider terminal communication link. The specific conversion process of the control instructions is a prior art means, for example, when the logistics safety decision indicates a risk warning, control instructions for triggering sound and light alarms are generated for the UAV and the rider terminal respectively and distributed. In addition, the UAV communication link and the rider terminal communication link can be realized by using existing wireless communication links.

[0060] In addition, the logistics safety monitoring system further includes, but is not limited to, a data recording platform communicatively connected to the multi-source video acquisition module, the AI intelligent analysis engine and the logistics safety decision center respectively, wherein the data recording platform is configured to collect and store the video stream data, the multi-modal recognition result and the logistics safety decision, so as to record and manage and / or trace back in history.

[0061] In summary, the logistics safety monitoring system based on AI video recognition technology provided by the embodiment has the following technical effects: (1) The embodiment provides a new logistics safety monitoring scheme designed for the cooperation scenario of UAV and rider, that is, a multi-source video acquisition module, an AI intelligent analysis engine, a logistics safety decision center and a cooperative control interface are sequentially communicatively connected, wherein the multi-source video acquisition module is configured to acquire video stream data of the article transfer process between the UAV and the rider from multiple angles in real time, the AI intelligent analysis engine is configured to perform multi-modal recognition analysis on the video stream data based on AI video recognition technology to obtain a multi-modal recognition result, the logistics safety decision center is configured to generate a logistics safety decision in real time according to the multi-modal recognition result, and the cooperative control interface is configured to convert the logistics safety decision into control instructions in real time and distribute the control instructions to the UAV and / or the rider terminal. Thus, through video intelligent recognition and analysis of the transferred articles, the rider and related environmental elements, the safety risks of the cooperation between the UAV and the rider can be effectively monitored, and corresponding decisions can be made to realize all-round protection of logistics safety, which is convenient for practical application and promotion.

[0062] Embodiment Two The embodiment further provides a working method of the logistics safety monitoring system based on AI video recognition technology according to the technical scheme of the first embodiment, as shown in Figure 2 The method includes, but is not limited to, the following steps S1-S4.

[0063] S1. The multi-source video acquisition module acquires video stream data of the article transfer process between the UAV and the rider from multiple angles in real time, wherein the multiple angles include a UAV side angle and a rider side angle.

[0064] S2. Real-time multi-modal recognition analysis of the video stream data is performed by an AI intelligent analysis engine based on AI video recognition technology to obtain multi-modal recognition results, wherein the multi-modal recognition results include handover article recognition results, rider behavior recognition results, man-machine legality recognition results, and handover environment recognition results.

[0065] S3. A logistics security decision center generates a logistics security decision in real time according to the multi-modal recognition results in combination with a preset article handover security rule and / or a multi-factor risk assessment model, wherein the logistics security decision includes any one of prohibition of handover, normal handover, suspension of handover, and risk warning.

[0066] S4. A cooperative control interface converts the logistics security decision into a control instruction in real time, and transmits the control instruction to the UAV in real time through a UAV communication link and / or transmits the control instruction to a rider terminal in real time through a rider terminal communication link.

[0067] The technical effects of the present embodiment can be conventionally deduced from the technical effects of Embodiment One, and will not be described here.

[0068] Finally, it should be noted that: the above only describes the preferred embodiments of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A logistics security monitoring system based on AI video recognition technology, characterized in that, The system comprises a plurality of video acquisition modules, an AI intelligent analysis engine, a logistics security decision center and a collaborative control interface connected in sequence. The plurality of video acquisition modules are configured to acquire video stream data of an article handover process between a UAV and a rider from multiple angles in real time, and transmit the video stream data to the AI intelligent analysis engine in real time. The AI intelligent analysis engine is configured to perform multi-modal recognition analysis on the video stream data in real time based on AI video recognition technology, obtain multi-modal recognition results, and transmit the multi-modal recognition results to the logistics security decision center in real time. The logistics security decision center is configured to generate a logistics security decision based on the multi-modal recognition results and in combination with a preset article handover security rule and / or a multi-factor risk assessment model in real time, and transmit the logistics security decision to the collaborative control interface in real time. The collaborative control interface is configured to convert the logistics security decision into a control instruction in real time, and transmit the control instruction to the UAV through a UAV communication link and / or to a rider terminal through a rider terminal communication link in real time.

2. The logistics security monitoring system of claim 1, wherein, The plurality of video acquisition modules comprise an onboard camera unit configured to acquire video stream data of the article handover process from a UAV side angle in real time, and a rider personal camera unit configured to acquire video stream data of the article handover process from a rider side angle in real time.

3. The logistics security monitoring system of claim 1, wherein, Based on AI video recognition technology, multi-modal recognition analysis is performed on the video stream data in real time to obtain multi-modal recognition results, including: Based on the video stream data, an article handover detection model pre-trained based on a YOLO target detection algorithm is used to detect the packaging integrity information of the article handover in real time, wherein the packaging integrity information includes packaging abnormal state based on the outer packaging video image detection of the article handover and used to indicate whether the outer packaging is damaged, deformed and / or opened; Based on the video stream data, a Deepsort algorithm is used to perform target tracking on the article handover to obtain the real-time motion trajectory of the article handover, and based on the real-time motion trajectory, it is judged whether there is an article throwing situation or a natural falling situation to obtain abnormal motion detection information of the article handover. Real-time integration of the package integrity information and the abnormal motion detection information to obtain handover article identification results.

4. The logistics security monitoring system of claim 1, wherein, Based on AI video recognition technology, real-time multi-modal recognition analysis is performed on the video stream data to obtain multi-modal recognition results, including: According to the video stream data, a rider detection model pre-trained based on a YOLO target detection algorithm and a human body action recognition model pre-trained based on a human body skeleton key point detection algorithm are used to identify the action behavior of the rider in real time, and the action behavior is matched with a preset normal behavior library and / or an abnormal behavior library to determine whether the action behavior belongs to a normal behavior or an abnormal behavior, thereby obtaining behavior state detection information; According to the video stream data, a Deepsort algorithm is used to track the rider in real time to obtain the real-time motion trajectory of the rider, and whether the change timing of the relative position and / or distance between the UAV and the rider conforms to a preset article handover safety specification is analyzed in real time based on the real-time motion trajectory, thereby obtaining human-machine interaction detection information; Real-time integration of the behavior state detection information and the human-machine interaction detection information to obtain rider behavior identification results.

5. The logistics security monitoring system of claim 1, wherein, Based on AI video recognition technology, real-time multi-modal recognition analysis is performed on the video stream data to obtain multi-modal recognition results, including: According to the video stream data, a gait feature recognition algorithm, a clothing feature recognition algorithm, and a delivery tool feature recognition algorithm are used to identify the gait feature, the clothing feature, and the delivery tool feature of the rider in real time, respectively, and the identity legitimacy of the rider is verified based on the gait feature, the clothing feature, and the delivery tool feature, thereby obtaining rider legitimacy verification information; According to the video stream data, a UAV visual feature recognition algorithm is used to identify the visual feature of the UAV in real time, and the handover task matching of the UAV and the rider is verified based on the visual feature and the gait feature, the clothing feature, and the delivery tool feature of the rider, thereby obtaining task matching verification information; Real-time integration of the rider legitimacy verification information and the task matching verification information to obtain human-machine legitimacy identification results.

6. The logistics security monitoring system of claim 1, wherein, Based on AI video recognition technology, real-time multi-modal recognition analysis is performed on the video stream data to obtain multi-modal recognition results, including: According to the video stream data, an obstacle detection model pre-trained based on a YOLO target detection algorithm is used to detect obstacles around the UAV in real time, and whether there is a safe place suitable for article handover directly below the UAV is analyzed in real time based on the obstacle detection result, thereby obtaining safe place detection information; According to the video stream data, a rider detection model and a personnel detection model pre-trained based on a YOLO target detection algorithm are used to detect intruding personnel directly below the UAV in real time, and the density, direction of movement, and / or distance of the intruding personnel to the UAV are analyzed in real time based on the intruding personnel detection result, thereby obtaining personnel intrusion detection information, wherein the intruding personnel refers to personnel other than the rider. According to the video stream data, real-time analysis obtains environment condition detection information of the surrounding environment of the UAV, wherein the environment condition detection information contains weather conditions and / or illumination conditions; Real-time integration of the safe place detection information, the personnel intrusion detection information and the environment condition detection information obtains an exchange environment recognition result.

7. The logistics security monitoring system of claim 1, wherein, According to the multi-modal recognition result, real-time generation of a logistics safety decision in combination with a preset article exchange safety rule, including: According to the multi-modal recognition result and the preset article exchange safety rule, if it is found in real time that there is no safe place suitable for article exchange directly below the UAV and / or the surrounding environment of the UAV is in a severe condition, a logistics safety decision for indicating prohibition of exchange is generated in real time; And / or, according to the multi-modal recognition result and the preset article exchange safety rule, if it is found in real time that there is a safe place suitable for article exchange directly below the UAV, the surrounding environment of the UAV is in a normal condition, the identity legality verification of the rider is passed and the exchange task matching verification between the UAV and the rider is passed, a logistics safety decision for indicating normal exchange is generated in real time; And / or, according to the multi-modal recognition result and the preset article exchange safety rule, if it is found in real time that the safe place directly below the UAV becomes a non-safe place, the surrounding environment of the UAV becomes a severe condition, the exchanged article appears an incomplete packaging condition / abnormal motion condition, the rider makes an abnormal behavior or there is an abnormal human-machine interaction condition between the UAV and the rider, a logistics safety decision for indicating suspension of exchange is generated in real time; And / or, according to the multi-modal recognition result and the preset article exchange safety rule, if it is found in real time that there is a personnel intrusion condition directly below the UAV, a logistics safety decision for indicating risk warning is generated in real time.

8. The logistics security monitoring system of claim 1, wherein, According to the multi-modal recognition result, real-time generation of a logistics safety decision in combination with a preset article exchange safety rule and a multi-factor risk assessment model, including: According to the multi-modal recognition result and the preset article exchange safety rule, real-time generation of an initial logistics safety decision, wherein the logistics safety decision includes any one of prohibition of exchange, normal exchange, suspension of exchange and risk warning; If the initial logistics safety decision does not indicate prohibition of exchange, according to the multi-modal recognition result and the preset multi-factor risk assessment model, real-time evaluation of four factor risk assessment scores corresponding to the exchange article recognition result, the rider behavior recognition result, the human-machine legality recognition result and the exchange environment recognition result respectively, and calculation of a risk assessment total score by using a weighted average method according to the four factor risk assessment scores; According to the risk assessment total score and a decision switching matrix included in the article handover safety rule and containing M×N elements, it is judged whether the initial logistics safety decision needs to be adjusted. If yes, the final logistics safety decision is obtained by adjustment, otherwise the initial logistics safety decision is directly taken as the final logistics safety decision, wherein M and N respectively represent positive integers, each row index of the decision switching matrix respectively corresponds to a logistics safety decision without indicating prohibition of handover, each column index of the decision switching matrix respectively corresponds to an evaluation interval, and the element value of the element is the unique number of the logistics safety decision.

9. The logistics security monitoring system of claim 1, wherein, Further comprising a data recording platform communicatively connected with the multi-source video acquisition module, the AI intelligent analysis engine and the logistics safety decision center respectively; The data recording platform is used for collecting and storing the video stream data, the multi-modal recognition result and the logistics safety decision.

10. A working method of the logistics safety monitoring system according to any one of claims 1-9, comprising: collecting video stream data of an article handover process between a UAV and a rider in real time from multiple angles by a multi-source video acquisition module, wherein the multiple angles include a UAV side angle and a rider side angle; performing multi-modal recognition analysis on the video stream data in real time based on AI video recognition technology by an AI intelligent analysis engine to obtain a multi-modal recognition result, wherein the multi-modal recognition result includes an article handover recognition result, a rider behavior recognition result, a man-machine legality recognition result and a handover environment recognition result; generating a logistics safety decision in real time by a logistics safety decision center according to the multi-modal recognition result in combination with a preset article handover safety rule and a multi-factor risk assessment model, wherein the logistics safety decision includes any one of prohibition of handover, normal handover, suspension of handover and risk warning; converting the logistics safety decision into a control instruction in real time by a cooperative control interface, and transmitting the control instruction to the UAV in real time through a UAV communication link and / or to a rider terminal in real time through a rider terminal communication link.

Images