A stereoscopic collaborative intelligent inspection method based on multi-source behavior perception

Abstract

This invention relates to the field of three-dimensional collaborative intelligent inspection technology. Specifically, it relates to a three-dimensional collaborative intelligent inspection method based on multi-source behavioral perception. It includes the following steps: S1, dividing the inspection area into inspection targets, collecting image data of the targets using unmanned vehicles and drones, and classifying the image data according to database standards; S2, determining the business status and classifying the image status based on the target's dedicated image database, while simultaneously connecting to security intelligent devices to establish a list of safe personnel; This invention utilizes a three-dimensional acquisition network encompassing ground, air, and indoor areas, with unmanned vehicles, drones, and security intelligent devices complementing each other, covering routine ground monitoring, enhanced aerial risk area acquisition, and indoor personnel verification, eliminating blind spots. Furthermore, it combines business status determination, regional data fusion, and damage feature comparison to focus on core risk scenarios during non-business hours.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional collaborative intelligent inspection technology, and more specifically, to a three-dimensional collaborative intelligent inspection method based on multi-source behavior perception. Background Technology

[0002] In the field of security monitoring and facility inspection, intelligent inspection technology serves as a key means to prevent abnormal risks and ensure regional security.

[0003] Currently, intelligent inspection technologies are mostly used in single, routine environments, such as counting people flow in commercial venues during business hours or inspecting the appearance of facilities in open, unobstructed outdoor areas. This can easily lead to blind spots and make it difficult to accurately identify hidden risks during non-business hours. Furthermore, the risk assessment mechanism is rigid, with existing technologies often using fixed thresholds that cannot be dynamically adjusted according to changes in the inspection path length or target security requirements. For example, a fixed failure threshold in long-path inspections can easily lead to missed risks, and a uniform personnel identification standard can easily misidentify frequently appearing unsafe personnel as normal personnel, or lack effective responses to special cases where facial recognition is not possible, thus reducing the accuracy of risk identification. Therefore, this paper proposes a three-dimensional collaborative intelligent inspection method based on multi-source behavior perception. Summary of the Invention

[0004] The purpose of this invention is to provide a three-dimensional collaborative intelligent inspection method based on multi-source behavior perception to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, a three-dimensional collaborative intelligent inspection method based on multi-source behavior perception is provided, comprising the following steps:

[0006] S1. Divide the inspection targets in the inspection area, collect image data of the inspection targets through unmanned vehicles and drones, and classify the image data according to the database standards;

[0007] S2. Determine the business status and classify the image status based on the exclusive image database of the inspection target, and connect to the security intelligent device to establish a list of security personnel.

[0008] S3. Determine the appearance and shape of the inspection target through non-business image data, combine the time series of image data with the appearance and shape of the inspection target, carry out regional image data fusion, and set a threshold based on the number of image data acquisition failures. Areas with more than the threshold of acquisition failures are designated as risk areas.

[0009] S4. Based on the non-business hours of the inspection target, dispatch drones to collect and update image data of the risk area, and combine the regional image data with historical regional image data to determine the damage characteristics.

[0010] S5. Based on the judgment result, identify the people around the inspection target, and then combine the distance data between the identified people and the inspection target with the matching results of the list of safe people to calculate the damage score of the person.

[0011] S6. Calculate the total damage score of surrounding people, classify them according to distance data, and drive them away with voice prompts. Accumulate the number of voice prompts and adjust the damage score of the person according to the corresponding standard based on the number of prompts.

[0012] As a further improvement to this technical solution, step S1 is as follows:

[0013] S1.1 Obtain the inspection route and geographical location of the inspection target of the unmanned vehicle. When the unmanned vehicle arrives at the location of the inspection target, it collects images of the inspection target through the camera device equipped on the unmanned vehicle.

[0014] S1.2 Establish a corresponding dedicated image database for the inspection target, and then the unmanned vehicle will classify and store the collected image data into the corresponding dedicated image database.

[0015] As a further improvement to this technical solution, the unmanned vehicle in S1 is an unmanned vehicle carrying a drone. Both the unmanned vehicle and the drone are equipped with camera devices to collect images. When condition S4 is triggered, the drone is started to carry out the data collection work, while the unmanned vehicle is responsible for real-time image acquisition.

[0016] As a further improvement to this technical solution, step S2 is as follows:

[0017] S2.1 Extract the latest image data corresponding to the inspection target's specific image data, compare the latest image data with historical image data, conduct scene feature difference analysis, identify the business status based on scene feature differences, divide the business status of the inspection target into operating and non-operating, and classify the image data according to the corresponding status of operating and non-operating to ensure that each type of image data corresponds to a clear operating or non-operating status.

[0018] S2.2. Collect indoor video of the inspection target through security intelligent equipment, extract facial features from the indoor video, set a threshold for the number of consecutive days of appearance, and then count the number of consecutive days of appearance of each facial feature based on the indoor video record. Compare the number of consecutive days of appearance of the facial feature with the threshold, and only retain facial features with a number of consecutive days of appearance greater than the threshold. Establish a list of safe persons based on the retained facial features.

[0019] S2.2.1 Set an expiration period for the list of safe persons. If a person’s facial feature has not appeared in the list for more than the expiration period, the facial feature will be removed from the list of safe persons.

[0020] As a further improvement to this technical solution, step S2 also includes the following steps:

[0021] S2.3 Mark the business status of the inspection targets based on the latest image data;

[0022] If the marked inspection target is in operation, no further steps are required.

[0023] If the target to be inspected is not in operation, continue with the subsequent steps in sequence.

[0024] S2.4 The security intelligent device has an alarm function. When the inspection target is not in a business state, if a person is identified in the indoor video and the person's facial features are not on the safe person list, an alarm will be triggered.

[0025] When a person's facial features cannot be extracted, the security smart device will issue a voice prompt. If the facial features still cannot be extracted after the voice prompt, an alarm will be triggered.

[0026] As a further improvement to this technical solution, step S3 is as follows:

[0027] S3.1 Divide the appearance of the inspection target into different regions, and fill the regions with the latest collected image data according to the time series of the image data.

[0028] S3.2. Based on the patrol path of the unmanned vehicle, set a threshold for the number of times the image data was acquired failed for the inspection target. Then, count the number of times the image data was acquired failed in each area and compare the number of acquisition failures with the corresponding failure threshold. If the number of acquisition failures exceeds the failure threshold, the area is determined to be in a risk state. Otherwise, if the number of acquisition failures does not exceed the failure threshold, monitoring continues.

[0029] The longer the patrol route, the lower the failure threshold.

[0030] The shorter the patrol route, the higher the failure threshold.

[0031] As a further improvement to this technical solution, step S4 is as follows:

[0032] S4.1 When the unmanned vehicle is collecting images of the inspection target, if the inspection target has a set risk area, it will dispatch a drone to fly to the inspection target and collect image data of the risk area through the drone's camera device, thereby updating the image data of the risk area.

[0033] S4.2 When the image data corresponding to the inspection target is updated, the latest updated image data of each area is combined with the historical image data with the most recent shooting time to perform damage feature analysis. If the image data is determined to have damage features, proceed to S5. Otherwise, if the image data is not determined to have damage features, continue monitoring.

[0034] As a further improvement to this technical solution, step S5 is as follows:

[0035] S5.1. By rotating the unmanned vehicle to adjust the acquisition angle of the camera device, the surrounding people and their facial features are collected, thereby establishing a list of surrounding people and the facial features of each surrounding person. Then, the facial features of each surrounding person are combined with the list of safe people for identification, and the surrounding people that match the list of safe people are removed from the list.

[0036] S5.2 Set risk distance levels for different distances, and calculate the straight-line distance data between each person in the surrounding personnel list and the inspection target. Then, combine the distance data of the surrounding personnel and the risk distance level to determine the level, and calculate the damage score of the personnel based on the risk distance level.

[0037] The higher the risk level, the higher the damage score;

[0038] S5.3 For people in the vicinity whose facial features cannot be obtained, set a risk level, and then compare the risk distance level of people in the vicinity whose facial features cannot be obtained with the risk level. If the risk distance level is at the risk level, drive them away directly.

[0039] As a further improvement to this technical solution, step S6 is as follows:

[0040] S6.1 Establish a database to store records of changes in the destruction scores of surrounding characters and to calculate the total destruction scores of surrounding characters.

[0041] A time decay mechanism is set for the character's total destruction score. If the character's destruction score does not increase for a long time, its total destruction score will decrease over time.

[0042] S6.2 Set a threshold for driving away scores, and then compare the total damage scores of all surrounding characters with the threshold for driving away scores;

[0043] When the total damage score exceeds the driving away score threshold, the drone will be used to drive away the people in the vicinity through intermittent voice reminders until the risk level corresponding to the distance between the people in the vicinity and the inspection target is reduced to the lowest level, and then the drone will return to the unmanned vehicle.

[0044] If the total damage score is less than the expulsion score threshold, monitoring continues.

[0045] S6.3 Count the number of times the surrounding characters are driven away by voice prompts, and set the increase range of the damage score corresponding to different number of voice prompts. Then update and increase the damage score according to the number of voice prompts for each character.

[0046] Set an alarm score threshold. When a character's damage score exceeds the alarm score threshold, an alarm will be triggered.

[0047] As a further improvement to this technical solution, S6.3 establishes a communication connection between the unmanned vehicle and the police system, sends an alarm reminder to the police system, and includes the location of the unmanned vehicle and the location of the patrol target. At the same time, it shares image data with the police system to assist police officers in their work.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] 1. In this three-dimensional collaborative intelligent inspection method based on multi-source behavior perception, a three-dimensional data collection network of ground, air and indoor areas is used. Unmanned vehicles, drones and security intelligent equipment complement each other and cover routine ground monitoring, enhanced data collection in high-risk areas in the air and personnel verification in the indoor area. There are no blind spots. At the same time, combined with business status determination, regional data fusion and damage feature comparison, it focuses on core risk scenarios during non-business hours. Through dynamic thresholds and dynamic maintenance of lists, risk areas and abnormal personnel are accurately identified.

[0050] 2. This three-dimensional collaborative intelligent patrol method based on multi-source behavioral perception establishes a hierarchical handling process of initial alarm, drone dispersal, and police linkage. Low-risk situations are continuously monitored, medium- and high-risk situations are dispersed by drones through intermittent voice communication, and ultra-high-risk situations trigger alarms and link with the police system, making the handling more targeted. At the same time, unmanned vehicles and drones can quickly coordinate. After a risk area is triggered, the drone immediately starts collecting and updating data, which solves the limitations of ground data collection and significantly shortens the risk response time. Moreover, through a scoring adjustment mechanism that links the number of voice reminders, high-risk personnel who ignore the dispersal can be quickly escalated to higher levels of handling, ensuring the safety of the patrol targets.

[0051] 3. In this three-dimensional collaborative intelligent inspection method based on multi-source behavior perception, the core parameters are dynamically adjusted, the failure threshold for data collection is flexibly changed with the length of the inspection path, the scoring decay mechanism avoids long-term misjudgment by personnel, the number of days threshold can be customized according to the safety requirements of the inspection target, adapts to different scenario risk control standards, and distinguishes between operating and non-operating states. In the operating state, the process is simplified and resource consumption is reduced, while in the non-operating state, full-process risk monitoring is initiated, which fits the actual inspection work logic and improves resource utilization efficiency. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a three-dimensional collaborative intelligent inspection method based on multi-source behavior perception according to the present invention.

[0053] Figure 2 This is a flowchart illustrating the process of obtaining the inspection route and geographical location of the inspection target for the unmanned vehicle in this invention.

[0054] Figure 3 This is a flowchart illustrating the process of extracting the latest image data corresponding to the specific image data of the inspection target in this invention.

[0055] Figure 4 This is a flowchart illustrating how the appearance of an inspection target is divided into different regions according to the present invention.

[0056] Figure 5 A flowchart illustrating the process of updating image data for risk areas in this invention;

[0057] Figure 6 This is a flowchart illustrating the process of adjusting the camera's viewing angle by rotating an unmanned vehicle, as described in this invention.

[0058] Figure 7 This is a flowchart illustrating the process of calculating the total damage score of surrounding individuals in this invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Please see Figure 1 - Figure 7 As shown, the purpose of this embodiment is to accurately identify the security risks of inspection targets, especially abnormal and destructive behaviors during non-business hours, through multi-device collaboration and multi-dimensional data fusion, and to form a closed-loop inspection system of data collection, status determination, risk location, enhanced collection, risk assessment, and graded handling. Unmanned vehicles are responsible for basic image collection and routine monitoring, drones focus on risk areas for enhanced collection, and security intelligent devices assist in personnel identity verification. The three work together to achieve three-dimensional perception of ground, air, and indoor environments. At the same time, through database linkage and scoring mechanisms, risk handling is made more targeted. Therefore, a three-dimensional collaborative intelligent inspection method based on multi-source behavioral perception is provided, including the following steps:

[0061] S1. Divide the inspection targets in the inspection area, collect image data of the inspection targets through unmanned vehicles and drones, and classify the image data according to the database standards;

[0062] To ensure the data source for the entire inspection process, the core is to use a vehicle-mounted drone-based hybrid unmanned vehicle to complete the targeted acquisition and dedicated storage of images of the inspection targets.

[0063] The steps for S1 are as follows:

[0064] S1.1 Obtain the inspection route and geographical location of the inspection target from the unmanned vehicle (UAV). When the UAV arrives at the inspection target location, it uses its onboard camera to capture images of the target. S1.2 Establish a dedicated image database for each inspection target. The UAV then categorizes and stores the collected image data into the corresponding dedicated image database. The specific steps are as follows:

[0065] First, the inspection route and the geographical location information of each inspection target are obtained from the unmanned vehicle control system. When the unmanned vehicle arrives at the location of the inspection target, it uses its equipped camera to collect images of the inspection target. The collected images must clearly show the appearance, key features and other information of the inspection target.

[0066] A dedicated image database is established for each inspection target to store all the images collected for that target. Then, the unmanned vehicle will classify and store the collected image data into the corresponding dedicated image database according to the different inspection targets, ensuring that each image can be accurately assigned to its corresponding inspection target database.

[0067] The S1 autonomous vehicle is an autonomous vehicle carrying a drone. Both the autonomous vehicle and the drone are equipped with camera devices to collect images. When the S4 condition is triggered, the drone is activated to carry out the data collection work, while the autonomous vehicle is responsible for real-time image collection.

[0068] S2. Determine the business status and classify the image status based on the exclusive image database of the inspection target, and connect to the security intelligent device to establish a list of security personnel.

[0069] The steps for S2 are as follows:

[0070] S2.1 Extract the latest image data corresponding to the inspection target's specific image data, compare the latest image data with historical image data, conduct scene feature difference analysis, and identify the business status based on scene feature differences. Divide the inspection target's business status into "open" and "non-open," and classify the image data accordingly to ensure that each type of image data corresponds to a clear "open" or "non-open" status. By comparing the features of new and old images, the business / non-open status is divided, achieving preliminary classification of the inspection scene and reducing unnecessary subsequent operations. The specific steps are as follows:

[0071] From the dedicated image database of each inspection target, the latest image data corresponding to each inspection target is extracted. This latest image data is then compared with the historical image data of that target to conduct scene feature difference analysis, identifying changes in scene features such as personnel activity, equipment operation, and environmental layout. Based on these scene feature differences, the business status of the inspection target is then identified, categorizing it as either open or closed. For example:

[0072] If the image shows characteristics such as people working normally, equipment operating, and business facilities being open, it is determined to be in operation.

[0073] If there is no human activity, equipment shutdown, or business facilities closed in the image, it is determined to be non-business.

[0074] The image data is categorized into operating and non-operating statuses to ensure that each category of image data corresponds to a clear operating or non-operating status. The categorized data is then stored separately for subsequent management and analysis.

[0075] S2.2. Indoor video of the inspection target is collected through security intelligent equipment. Facial features are extracted from the indoor video. At the same time, a threshold for the number of consecutive days of appearance is set (for example, it can be set to 7 days, which can be determined according to the security management needs of the inspection target). Then, based on the indoor video records, the number of consecutive days of appearance of each facial feature is counted, and the number of consecutive days of appearance of the facial feature is compared with the threshold. Only facial features with consecutive days of appearance greater than the threshold are retained. A list of safe persons is established based on the retained facial features (this list is used for subsequent security management of the indoor area of ​​the inspection target). The list of safe persons is dynamically maintained through the threshold for consecutive days of appearance of the face and the list expiration mechanism to ensure the timeliness and accuracy of the list.

[0076] S2.2.1 Set an expiration period for the list of safe personnel (for example, it can be set to 7 days, the specific number can be determined according to the safety management needs of the inspection target). When the number of consecutive days that a person’s facial feature has not appeared in the list of safe personnel exceeds the expiration period, the facial feature will be removed from the list of safe personnel.

[0077] S2 also includes the following steps:

[0078] S2.3 Mark the business status of the inspection targets based on the latest image data;

[0079] If the marked inspection target is in operation, no further steps are required.

[0080] If the target to be inspected is not in operation, continue with the subsequent steps in sequence.

[0081] S2.4 The security intelligent device has an alarm function. When the inspection target is not in a business state, if a person is identified in the indoor video and the person's facial features are not on the safe person list, an alarm will be triggered.

[0082] When a person's facial features cannot be extracted, the security smart device will issue a voice reminder. If the facial features still cannot be extracted after the voice reminder, an alarm will be triggered. Alarms will be triggered for abnormal personnel outside of business hours, forming the first line of defense. At the same time, considering the special case of facial features not being able to be extracted, the progressive approach of combining voice reminders with alarms will be used to avoid overlooking risks.

[0083] S3. Determine the appearance and shape of the inspection target through non-operational image data. Combine the time series of image data with the appearance and shape of the inspection target to carry out regional image data fusion. At the same time, set a threshold based on the number of image data acquisition failures and designate areas with more than the threshold as risk areas. Focus on the regional analysis of inspection targets in non-operational states and lock in risk areas through data fusion and threshold determination.

[0084] The steps for S3 are as follows:

[0085] S3.1 Divide the appearance of the inspection target into different areas, and fill the areas with the image data according to the time series, filling each area with the latest collected image data to ensure the integrity and timeliness of the image data of each area.

[0086] S3.2. Based on the patrol path of the unmanned vehicle, set a threshold for the number of times the image data was acquired failed for the inspection target. Then, count the number of times the image data was acquired failed in each area and compare the number of acquisition failures with the corresponding failure threshold. If the number of acquisition failures exceeds the failure threshold, the area is determined to be in a risk state. Otherwise, if the number of acquisition failures does not exceed the failure threshold, monitoring continues.

[0087] The longer the patrol route, the lower the failure threshold.

[0088] The shorter the patrol route, the higher the failure threshold.

[0089] By dynamically setting a threshold for failed data collection and adjusting the standard based on the length of the patrol path—the longer the path, the higher the requirement for a high success rate—this design aligns with actual inspection scenarios. It avoids missing risks due to multiple failed data collections during long-path inspections. At the same time, by comparing thresholds, it accurately marks abnormal data collection areas as risk areas, providing clear targets for subsequent enhanced data collection.

[0090] S4. Based on the non-business hours of the inspection target, dispatch drones to collect and update image data of the risk area, and combine the regional image data with historical regional image data to determine the damage characteristics; for special investigations of marked risk areas, the aerial perspective of drones makes up for the limitations of ground collection by unmanned vehicles.

[0091] The steps for S4 are as follows:

[0092] S4.1 When the unmanned vehicle is collecting images of the inspection target, if the inspection target has a set risk area, it immediately issues a dispatch command to send out the drone. After the drone quickly flies to the location of the inspection target, it uses its onboard camera to accurately collect image data of the risk area and replaces the old data of the risk area with the new data collected this time, thus updating the image data of the risk area; the activation conditions and core task of the drone are clearly defined, namely, updating the image data of the risk area, which solves the problem of unmanned vehicle collection failure.

[0093] S4.2 When the image data corresponding to the inspection target is updated, extract the latest updated image data for each area, and at the same time, filter out the historical image data of each area whose shooting time is closest to the latest data to prepare for subsequent comparative analysis. Combine the latest updated image data of each area with the historical image data with the closest shooting time to perform damage feature analysis (check whether there are features such as damage, missing parts, or deformation in the image that meet the preset damage standards). If the image data is determined to have damage features, proceed to S5; otherwise, if the image data is not determined to have damage features, continue monitoring.

[0094] By comparing the destructive features of new and old images, a secondary confirmation of the risk is completed. Only when destructive features are determined to exist will the subsequent processing procedure be initiated, avoiding excessive intervention in normal areas and ensuring the accuracy of risk identification.

[0095] S5. Based on the judgment result, identify the people around the inspection target, and then combine the distance data between the identified people and the inspection target with the matching results of the list of safe people to calculate the damage score of the person.

[0096] The steps for S5 are as follows:

[0097] S5.1. The camera's viewing angle is adjusted by rotating the unmanned vehicle to collect data on people around the inspection target and their facial features, thereby establishing a list of people in the vicinity and their facial features. Then, the facial features of each person in the vicinity are combined with the list of safe people for identification. People in the vicinity who match the list of safe people are removed from the list. The specific steps are as follows:

[0098] The unmanned vehicle flexibly adjusts its camera angle by rotating its onboard camera to ensure full coverage of the area surrounding the inspection target. With the help of the adjusted camera, it simultaneously captures images of people appearing around the inspection target and their corresponding facial features. During the acquisition process, it ensures that the images of people are clear and that their facial features are complete and identifiable. Then, based on the acquired information, it creates a list of people in the vicinity, clearly listing all the people appearing around the inspection target. It associates each person in the list with their corresponding facial features, forming a correspondence between people and facial features, which facilitates targeted identification in the future.

[0099] The facial features of each person in the list of surrounding people are extracted one by one. These facial features are then compared with the facial features in the established list of safe people to accurately check whether there is a match. After the comparison is completed, the surrounding people whose facial features match the list of safe people are selected and removed from the list of surrounding people. The information of the unmatched people is retained to provide a basis for further verification and other operations.

[0100] S5.2 Set risk distance levels for different distances, with close distance, medium distance, and long distance corresponding to different risk levels;

[0101] Within 3 meters is classified as Level 1 high risk, 3-5 meters as Level 2 medium risk, and more than 5 meters as Level 3 low risk.

[0102] Level 1 High Risk is the highest risk level, and the increase in damage score is also the highest.

[0103] Simultaneously, the straight-line distance data between each person in the surrounding list and the inspection target is calculated. Then, the distance data of the surrounding people and the risk distance level are combined to classify them. Based on the risk distance level, the damage score of the person is calculated.

[0104] The higher the risk level, the higher the damage score; establishing a correlation between distance and score aligns with the logic that closer distances mean higher risks in real-world scenarios, making the scoring more reasonable.

[0105] S5.3 For people in the vicinity whose facial features cannot be obtained, set a risk level (e.g., Level 1 High Risk), and then compare the risk distance level of people in the vicinity whose facial features cannot be obtained with the risk level. If the risk distance level is at the risk level, drive them away directly (drone voice prompt).

[0106] Additional rules are set up for special personnel whose faces cannot be obtained. By comparing the risk distance level with the set risk level, high-risk risks can be dealt with quickly, thus filling the gaps in the conventional scoring mechanism.

[0107] S6. Calculate the total damage score of people in the surrounding area, classify them according to distance data, and drive them away with voice reminders. Accumulate the number of voice reminders and adjust the damage score of the person according to the corresponding standard based on the number of reminders. In the closed-loop handling link of the entire inspection process, implement graded measures in combination with quantitative scores, and link with external systems to strengthen the handling efforts.

[0108] The steps for S6 are as follows:

[0109] S6.1 Establish a database to store records of changes in the destruction scores of surrounding characters and to calculate the total destruction scores of surrounding characters.

[0110] A time decay mechanism is implemented for a character's total damage score. If a character's damage score remains stagnant, their total damage score decreases over time. This ensures the objectivity of the scoring and avoids long-term misjudgments of characters. The specific steps are as follows:

[0111] First, a dedicated database is established to store records of damage rating changes for individuals in the surrounding area. Each record must contain specific information, such as the individual's unique identifier, the value of each damage rating, the time the rating was generated, and the reason for the rating change. This ensures that data storage is standardized and facilitates subsequent querying and statistics. Then, damage rating data for each individual in the surrounding area is extracted from the database and related data sources. All damage ratings for the same individual are aggregated and calculated to obtain the total damage rating for each individual. This total rating is then synchronously entered into the database and bound to the corresponding individual information.

[0112] Next, a time decay mechanism is set for the character's total destruction score, with clear decay rules, such as determining the decay period (e.g., one day as a decay period), the decay percentage or fixed decay value for each period. The core rule is that when a character's destruction score has not increased for a sustained period, the decay mechanism is triggered. Then, the changes in the destruction scores of each surrounding character are continuously monitored. If a character's destruction score has not increased for a set period, the character's total destruction score is reduced periodically over time according to the preset time decay mechanism. After each adjustment, the corresponding score change record in the database is updated in a timely manner. The formula is as follows: ;

[0113] Among them, S t S0 is the character's total damage score after time decay, n is the number of decay cycles in which the damage score does not increase, and λ is the decay coefficient for a single cycle (the value ranges from 0 to 1, and is set according to the risk tolerance of the inspection scenario; the higher the tolerance, the larger the coefficient).

[0114] S6.2 Set a driving away score threshold. Based on the actual situation such as the safety protection needs of the inspection target and the risk level of the surrounding environment, set a reasonable driving away score threshold. This threshold serves as the core judgment standard for determining whether it is necessary to drive away people in the surrounding area. Ensure that the threshold can effectively prevent risks without causing unnecessary driving away operations due to overly strict standards, such as a score of 40.

[0115] Then, the total destruction score of each surrounding person is compared with the driving away score threshold;

[0116] When the total damage score exceeds the driving away score threshold, a drone is used to drive away the target person by intermittent voice reminders (the drone activates its voice broadcasting device and uses intermittent voice reminders to drive away the target person; the content and interval of the voice reminders can be preset in advance to ensure the reminder effect while avoiding excessive interference). The drone will drive away the person in the vicinity until the risk level corresponding to the distance between the person in the vicinity and the inspection target is reduced to the lowest level. The intermittent voice reminders will then be stopped immediately. The drone will then receive a return command and fly back to the corresponding unmanned vehicle to complete the driving away mission.

[0117] High-scoring individuals are driven away using drones with intermittent voice commands, balancing effectiveness with human consideration.

[0118] When the total damage score is less than the deportation score threshold, monitoring continues, and the specific steps are as follows:

[0119] S6.3 Count the number of times the surrounding characters are driven away by voice prompts, and set the increase range of the damage score corresponding to different number of voice prompts. Then update and increase the damage score according to the number of voice prompts for each character.

[0120] Set an alarm scoring threshold (considering the importance of the inspection target, past risk event data, and security management needs, set a reasonable alarm scoring threshold. This threshold serves as the critical standard for triggering alarms, ensuring timely warnings of high-risk situations while avoiding frequent false alarms due to excessively low thresholds, such as a score of 80). When a person's damage score exceeds the alarm scoring threshold, an alarm is triggered. The specific steps are as follows:

[0121] Retrieve the operation records of the drones when they are carrying out the driving-away missions. For each person in the vicinity who is being driven away, count the number of voice reminders for them separately. During the counting process, ensure that the records are accurate and bind the number of times to the unique identifier of the corresponding person. Synchronously enter the data into the corresponding person information database. At the same time, in combination with the risk control standards of the inspection scenario, set corresponding rules for the number of voice reminders and the increase in the damage score.

[0122] Generally, the more reminders there are, the stronger the person's intention to ignore the expulsion, and the higher the corresponding increase. For example, 1-2 reminders correspond to a small increase, while 3 or more reminders correspond to a large increase. The specific increase standards for each number range are clearly defined.

[0123] Then, the system extracts the current damage score and corresponding number of voice reminders for each person being driven away from the database. Based on the previously set rules, it determines the appropriate increase in the damage score for that person, adds the increased score to their current damage score, updates the score, and promptly saves the updated score back to the database. The system periodically retrieves the updated damage scores of each person from the database and compares them one by one with the set alarm score thresholds. If a person's damage score exceeds the alarm score threshold, the system immediately triggers a preset alarm mechanism, such as sending an alarm message to the management personnel or activating on-site warning equipment. This ensures that if a person continues to ignore the driving away, an alarm is triggered, thus protecting the safety of the patrol target.

[0124] By adjusting the score based on the number of reminders, a linkage between handling, evaluation, and escalation is formed. At the same time, an alarm threshold is set, and when the score exceeds the threshold, it is linked to the police system, along with location and image data, to provide complete support for human intervention and achieve effective connection between intelligent patrol and offline handling.

[0125] S6.3 establishes a communication connection between the unmanned vehicle and the police system, sends alarm reminders to the police system, and includes the location of the unmanned vehicle and the location of the patrol target. At the same time, it shares image data with the police system to assist police officers in their work.

[0126] Through efficient equipment collaboration, unmanned vehicles and drones have a clear division of labor and complement each other's weaknesses, forming a three-dimensional data collection network. At the same time, the flexible mechanism design, such as dynamic thresholds, list invalidation, and score decay, allows inspection standards to be adapted to different scenarios. Furthermore, risk handling is tiered, from initial alarms to drone dispersal and then to police linkage, forming a graded handling system. Data linkage is achieved throughout the entire process, with dedicated databases, security equipment, and police systems connected to ensure seamless data flow from collection to application, significantly improving the intelligence and reliability of inspections.

[0127] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A stereoscopic collaborative intelligent inspection method based on multi-source behavior perception, characterized in that: It comprises the following steps: S1, dividing the inspection target in the inspection area, collecting the image data of the inspection target by the unmanned vehicle, and classifying the image data according to the database standard; S2, according to the exclusive image database of the inspection target, the business state is determined and the image state is classified, and the security intelligent device is connected, and the security person list is established through the security intelligent device; S3, the appearance shape of the inspection target is determined through the non-business image data, the time sequence of the image data is combined with the appearance shape of the inspection target, the regional image data fusion is carried out, and the threshold value is set according to the number of image data collection failures, and the area with the number of collection failures exceeding the threshold value is set as the risk area; S4, based on the non-business period of the inspection target, the unmanned aerial vehicle is dispatched to collect image data in the risk area, and the regional image data is combined with the historical regional image data to determine the damage characteristics; S5, according to the determination result, the surrounding people of the inspection target are identified, the distance data of the identified surrounding people and the inspection target are combined, and the matching result of the security person list is compared to calculate the damage score of the person; S6, the total damage score of the surrounding people is calculated, the distance data is classified by voice prompt, the number of voice prompts is accumulated, and the damage score of the person is adjusted according to the number corresponding standard.

2. The stereoscopic collaborative intelligent inspection method based on multi-source behavior perception according to claim 1, characterized in that: The steps of S1 are as follows: S1.1, the inspection route of the unmanned vehicle and the geographical position of the inspection target are obtained, when the unmanned vehicle arrives at the position of the inspection target, the image of the inspection target is collected by the camera device equipped on the unmanned vehicle; S1.2, the corresponding exclusive image database of the inspection target is established, and then the image data collected by the unmanned vehicle is classified and stored in the corresponding exclusive image database.

3. The method of claim 1, wherein the method is based on multi-source behavior perception and stereoscopic collaborative intelligent inspection. The unmanned vehicle of S1 is a unmanned vehicle carrying an unmanned aerial vehicle, and the unmanned vehicle and the unmanned aerial vehicle are both equipped with a camera device to collect images, when the condition of S4 is triggered, the unmanned aerial vehicle is started to collect the image, and the unmanned vehicle is responsible for real-time image collection.

4. The multi-source behavior perception based stereoscopic collaborative intelligent inspection method according to claim 1, characterized in that: The steps of S2 are as follows: S2.1, the latest image data of the exclusive image data corresponding to the inspection target is extracted, the latest image data and the historical image data are compared, the scene feature difference analysis is carried out, the business state is recognized according to the scene feature difference, the business state of the inspection target is divided into business and non-business, and the image data is classified according to the business and non-business, so that each kind of image data corresponds to a clear business or non-business state; S2.2, the indoor video of the inspection target is collected through the security intelligent device, the face features are extracted from the indoor video, a threshold value of continuous appearance days is set, the continuous appearance days of each face feature are counted according to the indoor video record, and the continuous appearance days of the face feature are compared with the threshold value of the number of days, only the face features with the continuous appearance days greater than the threshold value of the number of days are reserved, and the safety person list is established according to the reserved face features; S2.2.1, the invalid days of the safety person list are set, when the face features in the safety person list do not appear continuously for more than the invalid days, the face features are removed from the safety person list.

5. The multi-source behavior perception based stereoscopic collaborative intelligent inspection method according to claim 1, characterized in that: The S2 further comprises the following steps: S2.3, marking the business state of the inspection target according to the latest image data; When the marked inspection target is in the business state, the subsequent steps do not need to be executed; When the marked inspection target is in the non-business state, the subsequent steps are sequentially continued; S2.4, the security intelligent device has an alarm function, when the inspection target is in the non-business state, a person is identified in the indoor video, and the face features of the person are not in the safe person list, then the alarm is triggered; When the face features of the person cannot be extracted, the security intelligent device gives a voice reminder, and if the face features still cannot be extracted after the voice reminder, the alarm is triggered.

6. The multi-source behavior perception based stereoscopic collaborative intelligent inspection method according to claim 1, characterized in that: The steps of the S3 are as follows: S3.1, dividing the appearance shape of the inspection target into different regions, and filling the image data of the regions according to the time sequence, and filling the latest collected image data for each region; S3.2, combining the patrol path of the unmanned vehicle, setting the image collection failure number threshold for the inspection target, then counting the image data collection failure number of each region, and comparing the collection failure number with the corresponding failure number threshold, when the collection failure number exceeds the failure number threshold, it is determined that the region enters the risk state, otherwise, when the collection failure number does not exceed the failure number threshold, it is continuously monitored; The longer the patrol path, the lower the failure number threshold; The shorter the patrol path, the higher the failure number threshold.

7. The multi-source behavior perception based stereoscopic collaborative intelligent inspection method according to claim 1, characterized in that: The steps of the S4 are as follows: S4.1, when the inspection target has a set risk region during image collection by the unmanned vehicle, the unmanned vehicle is dispatched to fly to the inspection target, and the image data of the risk region is collected by the camera device of the unmanned vehicle to update the image data of the risk region; S4.2, when the corresponding image data of the inspection target is updated, the latest updated image data of each region is combined with the historical image data with the nearest shooting time to analyze the damage features, when it is determined that the image data has damage features, it enters S5, otherwise, when it is not determined that the image data has damage features, it is continuously monitored.

8. The multi-source behavior perception based stereoscopic collaborative intelligent inspection method according to claim 1, characterized in that: The steps of the S5 are as follows: S5.1, the collection angle of the camera device is adjusted by the unmanned vehicle rotation to collect the surrounding people and their face features of the inspection target, so as to establish a surrounding people list and the face features of each surrounding person, and then the face features of each surrounding person are combined with the safe person list for identification, and the surrounding people matching the safe person list are removed from the list; S5.2, setting different distance risk distance levels, and calculating the straight line distance data of each person in the surrounding people list from the inspection target, then combining the distance data of the surrounding people with the risk distance level for grading, and calculating the damage score of the person according to the risk distance level; The higher the risk distance level, the higher the damage score; S5.3, for the surrounding people who cannot obtain face features, set the risk level, then compare the risk distance level of the surrounding people who cannot obtain face features with the risk level, when the risk distance level is in the risk level, directly drive away.

9. The multi-source behavior perception based stereoscopic collaborative intelligent inspection method according to claim 1, characterized in that: The steps of the S6 are as follows: S6.1, establish a database for storing the damage score change record of the surrounding people, and count the total damage score of the surrounding people; Set a time decay mechanism for the total damage score of the character, and reduce the total damage score of the character according to the time when the damage score of the character does not increase continuously; S6.2, set a driving score threshold, and then compare the total damage score of each surrounding person with the driving score threshold; When the total damage score is greater than the driving score threshold, the unmanned aerial vehicle reminds the surrounding person through interval voice, and drives the surrounding person until the risk level corresponding to the distance between the surrounding person and the inspection target is reduced to the lowest, and then the unmanned aerial vehicle returns to the unmanned vehicle; When the total damage score is less than the driving score threshold, it is continuously monitored; S6.3, count the voice reminding times corresponding to the driving of the surrounding people, set the damage score up-regulation amplitude corresponding to different voice reminding times, and then update the damage score according to the voice reminding times corresponding to the character; Set an alarm score threshold, and when the damage score of the character exceeds the alarm score threshold, trigger the alarm.

10. The stereoscopic collaborative intelligent inspection method based on multi-source behavior perception according to claim 9, characterized in that: S6.3 establishes a communication connection between the unmanned vehicle and the police system, sends an alarm reminder to the police system, and attaches the position of the unmanned vehicle and the position of the inspection target, and shares image data with the police system, which provides assistance for the work of the police personnel.

Images