Real-time multi-target detection and behavior prediction method, terminal equipment and storage medium

Abstract

The invention is suitable for the field of image recognition, and discloses a real-time multi-target detection and behavior prediction method, terminal equipment and a storage medium. The real-time multi-target detection and behavior prediction method comprises the following steps: receiving video data acquired by an acquisition device in real time; performing multi-target real-time detection and real-time behavior prediction on the video data to obtain a behavior prediction result which comprises a target type, a target position and a potential risk; according to the behavior prediction result, a target robot is selected from a robot cluster, a control instruction set for the target robot is generated, and the control instruction set comprises a path guiding instruction and a security check reminding instruction; the control instruction set is sent to the target robot, and the target robot responds to the control instruction set and executes security check guiding operation. According to the invention, the security check response speed is effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, and particularly relates to a real-time multi-target detection and behavior prediction method, terminal device, and storage medium. Background Technology

[0002] Traditional security inspection systems often rely on single-modal video surveillance for initial target detection, only outputting detection results and unable to integrate with robot guidance. Furthermore, simple robot guidance only supports fixed voice announcements and cannot accurately schedule and guide targets based on real-time detected target types, locations, and potential risks. This results in the inability to quickly translate detection results into intervention actions during security checks, reducing efficiency. A new technological approach is needed to address these issues. Summary of the Invention

[0003] Therefore, embodiments of the present invention provide a real-time multi-target detection and behavior prediction method, terminal device, and storage medium, which can solve the problem of low security inspection efficiency in related technologies.

[0004] The first aspect of this invention provides a real-time multi-target detection and behavior prediction method, comprising: Receives video data collected in real time by the acquisition device; Multi-target real-time detection and real-time behavior prediction are performed on the video data to obtain behavior prediction results, wherein the behavior prediction results include target type, target location and potential risk; Based on the behavior prediction results, a target robot is selected from the robot cluster, and a set of control instructions for the target robot is generated, including path guidance instructions and security check reminder instructions. The control command set is sent to the target robot, and the target robot responds to the control command set by performing a security check guidance operation.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the step of selecting a target robot in the robot swarm based on the behavior prediction result includes: Retrieve the real-time position and status data of each robot in the robot cluster; Select the target robot that is closest to the target location and is in an idle state.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the step of selecting the target robot that is closest to and in an idle state based on the target location includes: Determine the distance between each candidate robot in the robot cluster and the target location, and filter out robots whose status data is marked as idle in the robot cluster to obtain a subset of candidate robots; The candidate robot with the smallest distance from the target position in the subset of candidate robots is selected as the target robot.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the step of generating a control instruction set for the target robot includes: Based on the target location and the preset security check location, an obstacle avoidance path is generated through a path planning algorithm, and the voice content is determined based on the target type. The obstacle avoidance path includes a coordinate sequence mapped to the robot's mobile coordinate system. Based on the obstacle avoidance path, the path guidance instruction is generated, and based on the voice content, the security check reminder instruction is generated, so as to obtain the control instruction set for the target robot.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, the step of selecting a target robot in the robot cluster based on the behavior prediction result and generating a control instruction set for the target robot includes: When the potential risk in the behavior prediction result is identified as a liquid or hazardous material, at least one target robot is selected in the robot cluster, and the set of control instructions for the target robot is generated; When the potential risk is zero, a release instruction is sent to the security inspection system, which responds to the release instruction and performs the release instruction operation.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the step of performing multi-target real-time detection and real-time behavior prediction on the video data to obtain a behavior prediction result, wherein the behavior prediction result includes target type, target location, and potential risk, includes: The pixel coordinates of the target in the image are located by using a target detection algorithm, and the pixel coordinates are mapped to the actual spatial location coordinates to obtain the target location; Based on the dangerous goods feature data transmitted in real time by the security inspection system, the contour shadow parameters of the target in the video data are compared, and the potential risks are identified.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, the step of sending the control instruction set to the target robot includes: The control instruction set is encrypted and packaged to obtain a data packet; When the LoRa signal strength of the target robot is lower than a preset threshold, the data packet is sent to the target robot via the WiFi 6 transmission channel. If the instruction confirmation information returned by the target robot is not received within a preset time period, the data packet is resent until the instruction confirmation information is received.

[0011] Optionally, in a seventh implementation of the first aspect of the present invention, the step of encrypting and packaging the control instruction set to obtain a data packet further includes: The control instruction set is encrypted to generate encrypted ciphertext; The decryption key matching the target robot is retrieved, and the decryption key and the encrypted ciphertext are packaged into the data packet. The decryption key is updated periodically and synchronized to the key storage unit of the robot cluster.

[0012] Secondly, embodiments of the present invention provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described real-time multi-target detection and behavior prediction method.

[0013] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described real-time multi-target detection and behavior prediction method.

[0014] Fourthly, embodiments of the present invention provide a computer program product that, when run on a terminal device, causes the terminal device to execute the aforementioned real-time multi-target detection and behavior prediction method.

[0015] The beneficial effects of this invention compared to existing technologies are as follows: By implementing a process of real-time data reception, detection and prediction, robot scheduling, and instruction execution, it solves the problems of disconnect between security inspection systems and robot guidance, and the inability to proactively intervene while only being able to passively detect. Directly linking the real-time detection of multiple targets with the behavior prediction results to the scheduling and control of the robot cluster enables seamless integration from target and risk detection to precise robot-guided security checks, effectively improving security inspection response speed and reducing security inspection delays caused by lag in manual guidance or disconnect between detection and intervention. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a schematic diagram of one embodiment of the real-time multi-target detection and behavior prediction method in this invention. Figure 2 This is a schematic diagram of a specific embodiment of step S103 of the real-time multi-target detection and behavior prediction method in this invention. Figure 3 This is a schematic diagram of a specific embodiment of step S1032 of the real-time multi-target detection and behavior prediction method in the present invention; Figure 4 This is a schematic diagram of a specific embodiment of step S102 of the real-time multi-target detection and behavior prediction method in the present invention; Figure 5 This is a schematic diagram of one embodiment of the terminal device in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are protected by this invention.

[0019] It should be noted that the terms "comprising," "including," and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this invention, are intended to cover non-exclusive inclusion. For example, a process, method, terminal, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. In the claims, specification, and accompanying drawings of this invention, relational terms such as "first" and "second" are used merely to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such immediate relationship or order between these entities / operations / objects.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] Traditional security inspection systems often rely on single-modal video surveillance for initial target detection, only outputting detection results and unable to integrate with robot guidance. Furthermore, simple robot guidance only supports fixed voice announcements and cannot accurately schedule and guide targets based on real-time detected target types, locations, and potential risks. This results in the inability to quickly translate detection results into intervention during security checks, reducing efficiency. A new technological approach is needed to address these issues.

[0022] In view of this, embodiments of the present invention provide a real-time multi-target detection and behavior prediction method, terminal device, and storage medium. Through a process of real-time data reception, detection prediction, robot scheduling, and instruction execution, it solves the problem of the disconnect between security inspection systems and robot guidance, and the inability to proactively intervene while only passively detecting. By directly linking the real-time multi-target detection and behavior prediction results to the scheduling and control of the robot cluster, a seamless connection can be achieved from target and risk detection to precise robot guidance for security inspections, effectively improving security inspection response speed and reducing security inspection delays caused by lag in manual guidance or the disconnect between detection and intervention.

[0023] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0024] Figure 1 This diagram illustrates a real-time multi-target detection and behavior prediction method according to an embodiment of the present invention. This method can be applied to terminal devices, such as mobile phones, tablets, laptops, ultra-mobile personal computers (UMPCs), and netbooks.

[0025] Specifically, the above-mentioned real-time multi-target detection and behavior prediction method may include the following steps S101 to S104.

[0026] Step S101: Receive video data collected in real time by the acquisition device.

[0027] The core functional modules of the terminal equipment include a data receiving module, a multi-target detection and behavior prediction module, a robot scheduling and control instruction generation module, and an instruction transmission module. These modules work together to realize the complete process from data collection to robot execution of security inspection guidance.

[0028] In an embodiment of the present invention, the terminal device receives video data transmitted in real time from a data acquisition device (such as a camera deployed at a subway station entrance) via a data receiving module.

[0029] Optionally, to enhance the data processing capabilities, the terminal device's data receiving module can simultaneously receive multimodal data, such as RGB video, depth images, or thermal imaging data.

[0030] Step S102: Perform multi-target real-time detection and real-time behavior prediction on the video data to obtain behavior prediction results, wherein the behavior prediction results include target type, target location and potential risk.

[0031] In an embodiment of the present invention, the multi-target detection and behavior prediction module of the terminal device processes the received real-time video data, identifies multiple types of targets in the video through a target detection algorithm, such as backpacks, water bottles, and suitcases carried by passengers; combines the target's movement trajectory, interaction status, and other information to perform real-time behavior prediction, and finally outputs behavior prediction results that include target type (e.g., backpack, thermos cup, suitcase), target location (e.g., specific spatial coordinates within a subway station entrance), and potential risks (e.g., no risk, suspected dangerous goods).

[0032] Optionally, the impact of environmental disturbances (such as crowd overlap and lighting changes) on the results can be reduced through adaptive attention mechanisms or multimodal fusion techniques.

[0033] Specifically, the adaptive attention mechanism addresses environmental variations in complex subway entrances, such as lighting changes, crowd overlap, and occlusion. It achieves precise focusing on effective features through dynamic weight allocation and multi-head attention division of labor. The dynamic weight allocation module adjusts the attention priority of feature map regions in real time based on scene complexity parameters such as light intensity, target occlusion rate, and crowd density, prioritizing the strengthening of feature weights valuable for target classification and localization. The multi-head attention division of labor module uses a parallel design to collect fine-grained object features (e.g., backpack compartments, water bottle markings), perform environmental noise filtering (e.g., reflections, strong light), collect target movement trajectories (e.g., rapid passenger movement), and detect interactions between targets (e.g., luggage collision trends). This addresses the shortcomings of existing technologies, such as insufficient fine-grained detection accuracy, poor dynamic adaptability, and weak robustness to environmental interference.

[0034] Specifically, multimodal fusion technology is used to fuse RGB, depth, thermal imaging, and audio multispectral features to obtain fused data. Specifically, the data layer synchronizes the multimodal acquisition timestamps and unifies the spatial coordinate system to achieve spatiotemporal alignment; the feature layer extracts the core information of each modality, completing redundancy removal and feature complementarity; the decision layer dynamically weights the detection results based on the confidence levels of each modality's features.

[0035] Optionally, when the edge computing server detects a suspicious target, it triggers the intelligent computing center to start a high-precision detection model.

[0036] Step S103: Based on the behavior prediction results, select a target robot in the robot cluster and generate a control instruction set for the target robot. The control instruction set includes path guidance instructions and security check reminder instructions.

[0037] In an embodiment of the present invention, the robot scheduling and control instruction generation module of the terminal device, based on behavior prediction results, selects target robots that meet the requirements from a preset robot cluster; then, based on information such as target type and potential risks, it generates a targeted set of control instructions. Among them, the path guidance instruction is used to guide the robot to the target location, and the security check reminder instruction is used to standardize the robot's communication of security check requirements to passengers (e.g., "Please show your belongings").

[0038] Optionally, to improve the rationality of robot selection, additional real-time status data of the robot cluster (such as robot idle status, busy status, and remaining battery power) can be retrieved, and the robot that is closest to the target location and is in an idle state can be selected first.

[0039] Step S104: The control command set is sent to the target robot, and the target robot responds to the control command set by performing a security check guidance operation.

[0040] In an embodiment of the present invention, the generated control instruction set is sent to the selected target robot in real time through the instruction transmission module; after receiving the instruction, the target robot responds to the control instruction set and performs the corresponding security check guidance operation (e.g., moving to the target passenger along the route planned by the path guidance instruction and broadcasting security check reminder instructions by voice or light).

[0041] Optionally, a LoRa / Wi-Fi dual-channel switching mechanism can be used to automatically switch to the Wi-Fi channel when the LoRa signal strength is insufficient, in order to avoid interruption of command transmission.

[0042] The beneficial effects of this invention compared to existing technologies are as follows: By implementing a process of real-time data reception, detection and prediction, robot scheduling, and instruction execution, it solves the problems of disconnect between security inspection systems and robot guidance, and the inability to proactively intervene while only being able to passively detect. Directly linking the real-time detection of multiple targets with the behavior prediction results to the scheduling and control of the robot cluster enables seamless integration from target and risk detection to precise robot-guided security checks, effectively improving security inspection response speed and reducing security inspection delays caused by lag in manual guidance or disconnect between detection and intervention.

[0043] In the robot guidance technology at subway entrances, the real-time location and status data of the robot cluster are not retrieved when selecting robots to perform security check guidance tasks. Instead, robots are selected through random assignment or fixed-order assignment, often resulting in the selection of robots that are far from the target location or are already busy, thus slowing down the overall security check process. Based on this, the present invention proposes an optional embodiment.

[0044] Reference Figure 2 , Figure 2This is a schematic diagram of a specific embodiment of step S103 of the real-time multi-target detection and behavior prediction method in this invention. Step S103 further includes the following specific implementation methods: Step S1031: Retrieve the real-time position and status data of each robot in the robot cluster.

[0045] In an embodiment of the present invention, a communication connection is established with the robot cluster through the robot status data retrieval module, and two core data of each robot in the cluster are retrieved in real time: one is real-time position data, such as the three-dimensional coordinates of the robot in the spatial coordinate system of the subway station entrance; the other is real-time status data, such as the calibration status of idle, busy, fault, etc., where the busy status includes situations such as the robot is performing a guidance task, charging, or being in a weak signal area.

[0046] Step S1032: Select the target robot that is closest to the target location and is in an idle state.

[0047] In an embodiment of the present invention, the target robot screening module of the terminal device extracts the target position from the behavior prediction results; performs double screening on the retrieved robot data, for example, first excluding robots that are busy or in a faulty state from all robots, and only retaining robots whose state is marked as idle; then, based on the preset distance calculation rules, calculates the straight-line distance between all idle robots and the target position, and finally selects the idle robot with the smallest distance as the target robot.

[0048] Optionally, if multiple idle robots are at the same distance from the target location, the robot with the higher remaining battery power will be selected first, based on the remaining battery power data of the robots.

[0049] In this embodiment of the invention, by retrieving robot position and status data in real time and prioritizing nearby idle robots, the problems of lack of targeted robot scheduling and delayed guidance response are solved.

[0050] In the robot guidance technology at subway entrances, when selecting robots to perform security check guidance tasks, only idle robots can be selected from the cluster. However, the distance between the idle robots and the target location is not calculated or sorted. This often results in random selection from idle robots, leading to situations where the selected robot is far from the target location, thus exacerbating the risk of security check delays. Based on this, the present invention proposes an optional embodiment.

[0051] Reference Figure 3 , Figure 3 This is a schematic diagram of a specific embodiment of step S1032 of the real-time multi-target detection and behavior prediction method in this invention. Step S1032 further includes the following specific implementation methods: Step S10321: Determine the distance between each candidate robot in the robot cluster and the target location, and filter out robots whose status data is marked as idle in the robot cluster to obtain a subset of candidate robots.

[0052] In an embodiment of the present invention, the terminal device retrieves the real-time position data of all robots in the robot cluster based on the target position obtained from the behavior prediction result. Using a preset distance algorithm, it calculates the straight-line distance between each robot and the target position, generating a list of robots and their corresponding distances. The robot status filtering module of the terminal device retrieves the real-time status data of all robots, removes robots that are not idle from the list, and retains only robots marked as idle and their corresponding distance data, forming a subset of candidate robots.

[0053] Optionally, the distance calculation module uses the Manhattan distance algorithm to correct the straight-line distance based on the characteristics of the subway station entrance scene, so that the calculation results are more consistent with the actual movement path of the robot; the robot status filtering module further excludes robots that are idle but have less than a preset battery level.

[0054] Step S10322: Select the candidate robot with the smallest distance from the target position from the subset of candidate robots as the target robot.

[0055] In an embodiment of the present invention, the target robot determination module of the terminal device sorts the robots and distance data in the candidate robot subset, extracts the robot with the smallest distance value in the sorting result, and marks it as the target robot for the final security check guidance task.

[0056] Optionally, if multiple robots in the candidate subset are at the same distance from the target location, the target robot determination module further retrieves the real-time movement speed data of these robots, and prioritizes the robot with the faster movement speed to shorten the time it takes for the robot to reach the target location.

[0057] In this embodiment of the invention, by first calculating the distance, then screening for available spaces, and finally selecting the closest one, the problem of low guidance efficiency caused by the robot screening only meeting the available space condition but not accurately matching the distance is solved.

[0058] Traditional voice broadcasts cause many non-target passengers to stop and look around, while the robot's fixed patrol route is often blocked by temporary obstacles. Based on this, the present invention proposes an alternative embodiment.

[0059] Step S103 further includes the following specific implementation methods: Step S1033: Based on the target location and the preset security check location, an obstacle avoidance path is generated through a path planning algorithm, and the voice content is determined through the target type. The obstacle avoidance path includes a coordinate sequence mapped to the robot's mobile coordinate system.

[0060] In an embodiment of the present invention, the terminal device first uses a path planning module and a preset path planning algorithm, combined with the real-time environmental characteristics of the subway station entrance, to generate an obstacle avoidance path containing a coordinate sequence mapped to the robot's movement coordinate system. This coordinate sequence clearly defines the robot's movement coordinates for each step from its current position to the target position and then to the security checkpoint. Simultaneously, the terminal device's voice content determination module matches targeted content from a preset voice template library based on the target type of the behavior prediction result (e.g., "backpack with a hidden compartment," "stainless steel thermos," "suitcase"). For example, when the target type is "stainless steel thermos," the determined voice content is "Please show your liquid container for security inspection"; when the target type is "backpack with a hidden compartment," the determined voice content is "Please open the backpack compartment for detailed inspection."

[0061] Optionally, the path planning module can combine the characteristics of a sudden increase in crowd density during peak hours, access crowd density data in real time, and dynamically adjust the width and turning nodes of the obstacle avoidance path.

[0062] Optionally, the voice content determination module can additionally associate the target with potential risks and add risk prompts to the voice, such as "Please confirm whether the liquid in the cup is drinking water. If it is not drinking water, further testing is required."

[0063] Step S1034: Generate the path guidance instruction based on the obstacle avoidance path, and generate the security check reminder instruction based on the voice content, so as to obtain the control instruction set for the target robot.

[0064] In an embodiment of the present invention, the control command generation module of the terminal device converts the obstacle avoidance path generated by the path planning module into a path guidance command that the robot can recognize. The command specifies the robot's movement direction, speed, and stopping point. The voice content matched by the voice content determination module is converted into a security check reminder command. The command includes the triggering time and volume of the voice playback. Finally, the control command generation module packages and integrates the path guidance command and the security check reminder command to form a complete control command set for the target robot.

[0065] Optionally, the control instruction generation module can add instruction execution time limit parameters to the instruction set.

[0066] In this embodiment of the invention, path planning based on obstacle information in the actual scene can control the robot's movement path error, which can effectively reduce the time passengers spend finding their way compared to traditional fixed route guidance; at the same time, based on customized voice commands with multi-target features, when there are many people waiting for security checks, the target can intuitively understand whether the person who needs to be checked is themselves.

[0067] Traditional security screening systems suffer from excessive interception, mechanically requiring all multi-target passengers to undergo baggage checks, which leads to the congestion of a large number of harmless passengers. Based on this, the present invention proposes an alternative embodiment.

[0068] Step S103 further includes the following specific implementation methods: Step S1035: When the potential risk in the behavior prediction result is identified as liquid or hazardous material, at least one target robot is selected in the robot cluster, and the control instruction set for the target robot is generated.

[0069] In an embodiment of the present invention, the internal mass density spectrum of multiple targets transmitted in real time by an X-ray device is received, and the contour parameters of the multiple targets obtained from video analysis are simultaneously compared. If the similarity between the contour shadow and the hazardous material feature library is greater than a preset percentage, the hazardous material risk is indicated; if the density value is between 1.0-1.5 g / cm³, the risk is indicated. 3 For liquids within a specified range and with a volume greater than 200ml, the risk of liquid calibration is considered.

[0070] Optionally, a whitelist of common items in the subway can be created to reduce false alarm rates.

[0071] Step S1036: When the potential risk is empty, a release instruction is sent to the security inspection system, and the security inspection system responds to the release instruction and performs the release instruction operation.

[0072] In an embodiment of the invention, for high-risk targets, the robot performs near-field guidance to the re-inspection area. For non-risk targets, a green passage signal is triggered, for example, through LED flashing and a prompting sound.

[0073] In this embodiment of the invention, by using a risk classification mechanism, robot guidance is initiated only for high-risk targets, which can improve the utilization rate of robot resources.

[0074] Traditional X-ray equipment, when operating alone, cannot correlate the appearance features of multiple targets, while pure video analysis is susceptible to interference from metallic reflections, leading to missed detections of dark-colored multi-targets. This fragmented detection method causes a surge in bag-opening rates during peak security checks. Based on this, the present invention proposes an alternative embodiment.

[0075] Reference Figure 4 , Figure 4 This is a schematic diagram of a specific embodiment of step S102 of the real-time multi-target detection and behavior prediction method in this invention. Step S102 further includes the following specific implementation methods: Step S1021: Locate the pixel coordinates of the target in the image using a target detection algorithm, and map the pixel coordinates to actual spatial location coordinates to obtain the target location.

[0076] In an embodiment of the invention, a wide-angle camera deployed above the security inspection machine captures images of multiple targets; a side-view depth camera supplements the occluded areas of the main viewpoint to generate a multi-target 3D point cloud. A pre-calibrated perspective transformation matrix is ​​loaded to transform the pixel coordinates of the multiple targets to the camera coordinate system; and the actual spatial location is matched using a SLAM map.

[0077] Optionally, Kalman filtering can be enabled to predict trajectories for multiple moving targets.

[0078] Step S1022: Based on the dangerous goods feature data transmitted in real time by the security inspection system, compare the contour shadow parameters of the target in the video data and mark the potential risks.

[0079] In an embodiment of the present invention, a density distribution matrix pushed per second by an X-ray device is received to identify potential risks.

[0080] In this embodiment of the invention, by fusing X-ray transmission data with anti-interference video analysis, the accuracy of liquid or knife identification can be significantly improved in subway security check areas containing metal reflective surfaces.

[0081] Traditional single-mode communication encounters multipath effects in subway turnstile areas, resulting in extremely high packet loss rates for robot commands. More seriously, the inability to detect packet loss and continued execution by default can cause robot stagnation or malfunctions. Based on this, the present invention proposes an optional embodiment.

[0082] Step S104 further includes the following specific implementation methods: Step S1041: Encrypt and package the control instruction set to obtain a data packet.

[0083] In an embodiment of the present invention, the control instruction set is encrypted.

[0084] Step S1042: When the LoRa signal strength of the target robot is lower than a preset threshold, switch to the WiFi 6 transmission channel to send the data packet to the target robot.

[0085] In an embodiment of the present invention, a real-time dashboard for robot communication quality is established. The LoRa signal strength of the target robot is collected every 200ms; when the RSSI is less than -90dBm for three consecutive times, a channel switching flag is triggered.

[0086] Optionally, the threshold can be dynamically adjusted based on the population density.

[0087] Step S1043: Receive instruction confirmation information returned by the target robot. If the instruction confirmation information is not received within a preset time period, resend the data packet until the instruction confirmation information is received.

[0088] In this embodiment of the invention, the command delivery rate can be significantly improved in subway stations with complex electromagnetic environments. Compared with traditional single-mode transmission, it can effectively avoid guidance interruption accidents caused by command loss.

[0089] Traditional robot systems use a pre-set unified key; if this key is leaked, the entire robot fleet will lose control. Based on this, the present invention proposes an alternative embodiment.

[0090] Step S1041 further includes the following specific implementation methods: Step S10411: Encrypt the control instruction set to generate encrypted ciphertext.

[0091] Step S10412: Retrieve the decryption key that matches the target robot, and package the decryption key and the encrypted ciphertext into the data packet. The decryption key is updated periodically and synchronized to the key storage unit of the robot cluster.

[0092] In this embodiment of the invention, by using key encapsulation and physical storage, data hijacking attempts can be resisted even in the public network environment of a subway station.

[0093] like Figure 5 The diagram illustrates a terminal device according to an embodiment of the present invention. The terminal device 500 may include a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501, such as a real-time multi-target detection and behavior prediction program. When the processor 501 executes the computer program 503, it implements the steps described in the various real-time multi-target detection and behavior prediction embodiments.

[0094] A computer program can be divided into one or more modules / units. One or more modules / units are stored in memory 502 and executed by processor 501 to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a terminal device.

[0095] The terminal device may include, but is not limited to, processor 501 and memory 502. Those skilled in the art will understand that... Figure 5 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, a terminal device may also include input / output devices, network access devices, buses, etc.

[0096] The processor 501 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0097] The memory 502 can be an internal storage unit of the terminal device, such as the hard drive or RAM of the terminal device. The memory 502 can also be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 502 can include both internal and external storage units of the terminal device. The memory 502 is used to store computer programs and other programs and data required by the terminal device. The memory 502 can also be used to temporarily store data that has been output or will be output.

[0098] It should be noted that, for the sake of convenience and brevity, the structure of the terminal device described above can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0099] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described real-time multi-target detection and behavior prediction method.

[0100] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to perform the steps in the aforementioned real-time multi-target detection and behavior prediction method.

[0101] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0102] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this invention.

[0103] In the embodiments provided by this invention, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0107] The embodiments described above are merely illustrative of the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A real-time multi-target detection and behavior prediction method, characterized in that, The method comprises the following steps: receiving video data collected by a collection device in real time; performing multi-target real-time detection and real-time behavior prediction on the video data to obtain a behavior prediction result, wherein the behavior prediction result comprises a target type, a target position, and a potential risk; selecting a target robot from a robot cluster according to the behavior prediction result, and generating a control instruction set for the target robot, wherein the control instruction set comprises a path guiding instruction and a security check reminding instruction; sending the control instruction set to the target robot, and causing the target robot to perform a security check guiding operation in response to the control instruction set.

2. The real-time multi-target detection and behavior prediction method of claim 1, wherein, The step of selecting a target robot from a robot cluster according to the behavior prediction result comprises: calling real-time position and state data of each robot in the robot cluster; selecting the target robot that is closest to the target position and in an idle state according to the target position.

3. The real-time multi-target detection and behavior prediction method of claim 2, wherein, The step of selecting the target robot that is closest to the target position and in an idle state according to the target position comprises: determining distances between each candidate robot in the robot cluster and the target position, and screening out robots in an idle state from the robot cluster to obtain a candidate robot subset; selecting a candidate robot that is closest to the target position from the candidate robot subset as the target robot.

4. The real-time multi-target detection and behavior prediction method of claim 1, wherein, The step of generating a control instruction set for the target robot comprises: generating an obstacle avoidance path by a path planning algorithm according to the target position and a preset security check position, and determining voice content according to the target type, wherein the obstacle avoidance path comprises a coordinate sequence mapped to a robot movement coordinate system; generating the path guiding instruction according to the obstacle avoidance path, and generating the security check reminding instruction according to the voice content, to obtain the control instruction set for the target robot.

5. The real-time multi-target detection and behavior prediction method of claim 1, wherein, The step of selecting a target robot from a robot cluster according to the behavior prediction result, and generating a control instruction set for the target robot comprises: when the potential risk in the behavior prediction result is liquid or hazardous material, selecting at least one target robot from the robot cluster, and generating the control instruction set for the target robot; when the potential risk is empty, sending a release instruction to a security check system, and causing the security check system to perform a release indicating operation in response to the release instruction.

6. The real-time multi-target detection and behavior prediction method of claim 1, wherein, The step of performing multi-target real-time detection and real-time behavior prediction on the video data to obtain a behavior prediction result, wherein the behavior prediction result comprises a target type, a target position, and a potential risk comprises: locating a pixel coordinate of a target in an image by a target detection algorithm, and mapping the pixel coordinate to an actual space position coordinate to obtain the target position; comparing a contour shadow parameter of the target in the video data with hazardous material feature data transmitted by a security check system in real time, and labeling the potential risk.

7. The real-time multi-target detection and behavior prediction method of claim 1, wherein, The step of sending the control instruction set to the target robot comprises: encrypting and packaging the control instruction set to obtain a data packet; Switch to a WiFi6 transmission channel to send the data packet to the target robot when the LORA signal strength of the target robot is lower than a preset threshold; Receiving instruction confirmation information returned by the target robot, and if the instruction confirmation information is not received within a preset time length, the data packet is re-sent until the instruction confirmation information is received.

8. The real-time multi-target detection and behavior prediction method of claim 7, wherein, The step of encrypting and packaging the control instruction set to obtain the data packet further comprises: Encrypting the control instruction set to generate an encrypted ciphertext; Accessing a decryption key matched with the target robot, and packaging the decryption key and the encrypted ciphertext as the data packet, wherein the decryption key is updated regularly and synchronized to a key storage unit of the robot cluster.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the real-time multi-target detection and behavior prediction method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the real-time multi-target detection and behavior prediction method according to any one of claims 1 to 8.

Images