Inspection method and system for military inspection device

Abstract

The present invention belongs to the technical field of inspection methods, and discloses an inspection method and system for a military inspection device. The method includes: obtaining a number of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data; using the first historical image data and the historical three-dimensional point cloud data as input to train a machine learning calibration model until the error between the output and the corresponding second historical image data is less than an error threshold to obtain a target calibration model; obtaining measured image data based on a camera, and obtaining measured three-dimensional point cloud data based on a laser radar; and inputting them into a target calibration model in sequence to obtain target image data; splicing the target image data to obtain a three-dimensional inspection map and loading it into a military inspection device so that it can inspect the inspection area to obtain target image data, and inputting the target image data into a target recognition model to determine whether there is an abnormality. The present invention has the advantage of high accuracy when conducting inspections.

Description

Technical Field

[0001] The present invention relates to the technical field of inspection methods, and in particular to an inspection method and system for a military inspection device. Background Art

[0002] In scenarios such as people's daily activities, industrial production, and national defense, inspecting the environmental status or facility deployment is the key to ensuring the smooth and safe development and operation of various scenarios.

[0003] Traditional inspections are mostly based on regular manual patrol inspections. However, this type of inspection method not only has high labor costs but also low inspection efficiency. It is also unable to quickly obtain operational anomalies, resulting in greater harm when anomalies occur. Therefore, manual inspections are gradually being replaced by intelligent inspection methods based on inspection devices. Specifically, this type of inspection method mainly includes the following steps: First, the inspection path is set and input into the inspection device; second, the inspection device selects the inspection type based on the inspection path. Specifically, it can choose to conduct automatic inspections from the starting point of the path, or from any random point selected on the path, or conduct inspections by manual control, etc.; then, based on the selected inspection type, the inspection device uses integrated cameras, radars and other detection equipment to check the environmental parameters, equipment parameters and equipment operating status of each path; finally, the inspection results are analyzed and transmitted to the remote control terminal, or the inspection results are directly transmitted to the remote control terminal in real time to complete the inspection process.

[0004] Therefore, the entire inspection process based on inspection devices must be based on inspection routes. However, existing inspection routes are only obtained using common surveying methods such as total stations, without considering the accuracy flaws of inspection routes obtained using these methods. However, in some scenarios, such as military applications such as national defense, more precise and accurate inspections are required to avoid inspection errors such as missed or false detections. Summary of the Invention

[0005] The object of the present invention is to provide an inspection method and system for a military inspection device, so as to improve the technical problems such as low accuracy of the inspection process of the current inspection device, resulting in information omission.

[0006] To achieve the above objectives, the present invention proposes the following technical solutions:

[0007] In the first aspect, this technical solution provides an inspection method for a military inspection device, including:

[0008] Acquiring a plurality of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data based on a historical database; wherein the first historical image data is image data with occlusion under abnormal conditions, the second historical image data is image data with clarity under normal conditions, and the historical three-dimensional point cloud data corresponds to the occluded areas in the first historical image data;

[0009] Iteratively training a machine learning calibration model using the first historical image data and the historical three-dimensional point cloud data as input until an error between an output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model;

[0010] Acquire a plurality of measured image data of the target area based on a camera mounted on the aerial survey equipment, and simultaneously acquire a plurality of measured three-dimensional point cloud data of the target area based on a laser radar mounted on the aerial survey equipment; and sequentially input the plurality of measured image data and the plurality of measured three-dimensional point cloud data into the target calibration model to acquire a plurality of target image data;

[0011] The target image data are spliced ​​to obtain a three-dimensional inspection map and loaded into a military inspection device so that it can inspect the inspection area to obtain a number of target image data, and the target image data are input into a target recognition model to determine whether there is an abnormality; wherein, the target recognition model is loaded on the military inspection device.

[0012] Furthermore, the training process of the target recognition model includes:

[0013] Obtain some historical image data to construct training sets and validation sets;

[0014] Training the initial recognition model based on the training set, and verifying the trained initial recognition model based on the verification set;

[0015] Repeat the previous step to iteratively train the initial recognition model until the target recognition model is obtained.

[0016] Furthermore, after inputting the target image data into a target recognition model to determine whether there is an abnormality, the method includes:

[0017] When it is determined that the abnormality corresponding to the target image data is the presence of an uncontrolled device or person in the target area, random disturbance is added to the target image data and then sent to the background control end, and at the same time, a broadcast is broadcast to the uncontrolled device to make it leave the target area;

[0018] When it is determined that the abnormality corresponding to the target image data is an abnormal operation of the equipment deployed in the target area, the abnormally operating equipment is continuously observed and inspected for a preset number of times; if each observation and inspection is determined to be an equipment abnormality, random disturbance is added to the target image data and then sent to the background control end.

[0019] Furthermore, the method of splicing the plurality of target image data to obtain a three-dimensional inspection map and loading the map into a military inspection device so that the device can inspect the inspection area to obtain a plurality of target image data includes:

[0020] Obtain a list of abnormal devices within the previous time period, including the abnormal device number, abnormal device location, and abnormal device frequency;

[0021] An inspection path is planned within the three-dimensional inspection map based on the abnormal equipment list; and an inspection is performed based on the inspection path.

[0022] Further, including:

[0023] When it is determined that the equipment deployment or geographical features of the inspection area are inconsistent with the corresponding area in the three-dimensional inspection map, an update message is sent to the remote control terminal so that the remote control terminal can obtain updated image data and updated three-dimensional point cloud data of the corresponding area based on the aerial survey equipment;

[0024] Acquire an updated local three-dimensional map based on the updated image data and the updated three-dimensional point cloud data according to the method;

[0025] The three-dimensional inspection map is updated based on the local three-dimensional map and then reloaded into the military inspection device.

[0026] In the second aspect, this technical solution provides an inspection system for a military inspection device. It includes:

[0027] A first acquisition module is configured to acquire a plurality of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data based on a historical database; wherein the first historical image data is image data with occlusion under abnormal conditions, the second historical image data is image data with clarity under normal conditions, and the historical three-dimensional point cloud data corresponds to the occluded areas in the first historical image data;

[0028] a model training module, configured to iteratively train a machine learning calibration model using the first historical image data and the historical three-dimensional point cloud data as inputs until an error between an output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model;

[0029] The second acquisition module is configured to acquire a plurality of measured image data of the target area based on a camera mounted on the aerial survey equipment, and simultaneously acquire a plurality of measured three-dimensional point cloud data of the target area based on a laser radar mounted on the aerial survey equipment; and sequentially input the plurality of measured image data and the plurality of measured three-dimensional point cloud data into the target calibration model to acquire a plurality of target image data;

[0030] The actual inspection module is used to splice the multiple target image data to obtain a three-dimensional inspection map and load it into the military inspection device so that it can inspect the inspection area to obtain multiple target image data, and input the target image data into a target recognition model to determine whether there is an abnormality; wherein, the target recognition model is loaded on the military inspection device.

[0031] Further, including:

[0032] A first judgment module is configured to, when determining that the abnormality corresponding to the target image data is the presence of an uncontrolled device or person in the target area, add random perturbations to the target image data and send the result to the backend control terminal, while simultaneously broadcasting to the uncontrolled device to cause it to leave the target area;

[0033] The second judgment module is used to judge that when the abnormality corresponding to the target image data is an abnormal operation of the equipment deployed in the target area, the equipment with abnormal operation is continuously observed and inspected for a preset number of times; if each observation and inspection is judged to be an equipment abnormality, random disturbance is added to the target image data and then sent to the background control end.

[0034] Furthermore, the actual inspection module includes:

[0035] An acquisition unit is used to acquire a list of abnormal devices in a previous time period, wherein the list of abnormal devices includes the abnormal device number, abnormal device location, and abnormal device frequency;

[0036] A planning unit is used to plan an inspection path within the three-dimensional inspection map based on the abnormal equipment list; and perform inspection based on the inspection path.

[0037] In a third aspect, the present technical solution provides an electronic device comprising at least one processor coupled to a memory, wherein a computer program is stored in the memory, and the computer program is configured to execute the method when executed by the processor.

[0038] In a fourth aspect, the present technical solution provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to execute the method described.

[0039] Beneficial effects:

[0040] It can be seen from the above technical solutions that the technical solution of the present invention provides an inspection method for a military inspection device to improve the defect of low inspection accuracy.

[0041] The method includes: obtaining a number of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data based on a historical database; wherein, the first historical image data is image data with occlusion in an abnormal environment, the second historical image data is image data that is clear in a normal environment, and the historical three-dimensional point cloud data corresponds to the occluded area in the first historical image data. The first historical image data and the historical three-dimensional point cloud data are used as input to iteratively train a machine learning calibration model until the error between the output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model. Based on the camera installed on the aerial survey equipment, a number of measured image data of the target area is obtained, and at the same time, based on the laser radar installed on the aerial survey equipment, a number of measured image data of the target area is obtained; and the number of measured image data and the number of measured three-dimensional point cloud data are sequentially input into the target calibration model to obtain a number of target image data. The target image data are spliced ​​to obtain a three-dimensional inspection map and loaded into a military inspection device so that it can inspect the inspection area to obtain a number of target image data, and the target image data are input into a target recognition model to determine whether there is an abnormality; wherein, the target recognition model is loaded on the military inspection device.

[0042] It can be seen that this technical solution is applicable to military inspection devices, so its inspection objects include not only deployed equipment, but also geographical representations; and inspections are mostly carried out outdoors. Furthermore, the inspection map construction involved in this technical solution is based on the image data captured by the aerial survey equipment. However, in the actual shooting process, affected by the weather environment, such as rainy, snowy, foggy, etc., the captured image data often cannot accurately reflect the real situation. LiDAR has the advantage of strong penetration when acquiring data. Therefore, this embodiment introduces three-dimensional point cloud data and a machine learning calibration model to calibrate the image data with occlusion. And because the introduced three-dimensional point cloud data only corresponds to the occluded area, it avoids the defect of a cumbersome process caused by the collection of a large amount of point cloud data. At this time, the target calibration model obtained based on the training can obtain target image data with accurate information expression; and then the three-dimensional inspection map constructed based on it has the advantage of high accuracy.

[0043] At the same time, since the military inspection device is equipped with a target recognition model for specific inspections, it not only improves the timeliness of inspections, but also helps improve the accuracy of inspections. Furthermore, since the three-dimensional inspection map is obtained by splicing, it is easy to update the three-dimensional inspection map, so as to facilitate more accurate inspections.

[0044] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, to the extent such concepts are not mutually inconsistent, can be considered to be part of the inventive subject matter of this disclosure.

[0045] The foregoing and other aspects, embodiments, and features of the present invention will be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as features and / or beneficial effects of the exemplary embodiments, will become apparent from the following description or through practice of specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For the sake of clarity, not every component is labeled in every figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings, in which:

[0047] Figure 1 This is a flow chart of the inspection method for the military inspection device described in this embodiment;

[0048] Figure 2 This is a flowchart for obtaining the target recognition model described in this embodiment;

[0049] Figure 3 This is a flowchart for updating the three-dimensional inspection map described in this embodiment;

[0050] Figure 4 This is a flowchart of exception handling in this embodiment;

[0051] Figure 5 This is a flow chart for setting the inspection path in this embodiment;

[0052] Figure 6 This is a structural block diagram of the inspection method for a military training device according to this embodiment;

[0053] Figure 7 Schematic diagram of the structure of the electronic device described in this embodiment. DETAILED DESCRIPTION

[0054] To further clarify the objectives, technical solutions, and advantages of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only a portion of the embodiments of the present invention, not all of them. Based on the described embodiments of the present invention, all other embodiments derived by persons of ordinary skill in the art without requiring creative effort are within the scope of protection of the present invention. Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meanings understood by persons of ordinary skill in the field to which the present invention pertains.

[0055] The words "first", "second" and similar terms used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "a", "an" or "the" and similar terms do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before "include" or "comprise" include the features, wholes, steps, operations, elements and / or components listed after "include" or "comprise", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0056] As is well known, the entire inspection process using an inspection device requires an inspection path as its foundation. However, existing inspection paths are obtained solely through common surveying methods, such as total stations, without taking into account the accuracy limitations of such surveying methods. However, in some scenarios, such as military applications like national defense, more precise and accurate inspections are required to avoid inspection errors, such as missed or false detections. Therefore, this embodiment aims to provide an inspection method for a military inspection device to address these technical limitations.

[0057] The inspection method of the military inspection device described in this embodiment will be described in detail below with reference to the accompanying drawings.

[0058] like Figure 1 As shown, the method includes:

[0059] Step S202: Acquire a number of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data based on a historical database.

[0060] In this embodiment, the first historical image data is image data with occlusion in an abnormal environment, the second historical image data is clear image data in a normal environment, and the historical three-dimensional point cloud data corresponds to the occluded area in the first historical image data.

[0061] Since this embodiment is applicable to military inspection devices, its inspection objects include not only deployed equipment but also geographical representations; and inspections are mostly conducted outdoors. Furthermore, the inspection map of this embodiment is constructed based on the image data captured by the aerial survey equipment. However, in the actual shooting process, due to the influence of the weather environment, such as rainy, snowy, cloudy and foggy weather, the captured image data often cannot accurately reflect the real situation. LiDAR has the advantage of strong penetration when acquiring data. Therefore, this embodiment introduces three-dimensional point cloud data to calibrate the image data with occlusion. And because the introduced three-dimensional point cloud data only corresponds to the occluded area, it avoids the defect of cumbersome process caused by the collection of a large amount of point cloud data.

[0062] At this point, the following steps will be performed:

[0063] Step S204: Iteratively train a machine learning calibration model using the first historical image data and the historical three-dimensional point cloud data as input until the error between the output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model.

[0064] In this embodiment, the machine learning calibration model is specifically an XGBoost regression model; the error threshold is specifically 0.9; and the stopping condition for iterative training may also be based on reaching a preset number of iterations.

[0065] At this point, a target calibration model will be obtained based on steps S202 to S204. Therefore, the three-dimensional inspection map will be obtained through the following steps:

[0066] Step S206: Acquire a number of measured image data of the target area based on the camera mounted on the aerial survey equipment, and simultaneously acquire a number of measured three-dimensional point cloud data of the target area based on the laser radar mounted on the aerial survey equipment; and input the number of measured image data and the number of measured three-dimensional point cloud data into the target calibration model in sequence to obtain a number of target image data.

[0067] In this embodiment, to facilitate data acquisition, a camera for acquiring image data and a lidar for acquiring 3D point cloud data are both installed on the aerial survey equipment. In practice, when acquiring measured image data, it is first analyzed and determined to identify occluded areas. Then, the lidar is used to acquire measured 3D point cloud data for these occluded areas. Specifically, the measured image data can be analyzed and determined based on a pre-trained image detection model to identify occluded areas.

[0068] Step S208: splicing the target image data to obtain a three-dimensional inspection map and loading it into a military inspection device to inspect the inspection area to obtain target image data, and inputting the target image data into a target recognition model to determine whether there is an abnormality.

[0069] As a specific implementation method, Figure 2 As shown, the target recognition model is obtained through the following steps:

[0070] Step S302: Acquire some historical image data to construct a training set and a validation set.

[0071] In this embodiment, the training set and the validation set are obtained by random division.

[0072] Step S304: train the initial recognition model based on the training set, and verify the trained initial recognition model based on the verification set.

[0073] Step S306: Repeat the previous step to iteratively train the initial recognition model until the target recognition model is obtained.

[0074] In this embodiment, the target recognition model is also loaded on the military inspection device. Since the three-dimensional inspection map is obtained by splicing, it is convenient to update the three-dimensional inspection map when the geographical representation or equipment deployment changes. As a specific implementation method, Figure 3 As shown, the map is updated as follows:

[0075] Step S402: When it is determined that the equipment deployment or geographical features of the inspection area are inconsistent with the corresponding area in the three-dimensional inspection map, update information is sent to the remote control terminal so that it can obtain updated image data and updated three-dimensional point cloud data of the corresponding area based on the aerial survey equipment.

[0076] Step S404: obtaining an updated local three-dimensional map based on the updated image data and the updated three-dimensional point cloud data according to the method described above.

[0077] Step S406: updating the three-dimensional inspection map based on the local three-dimensional map and reloading it into the military inspection device.

[0078] In summary, this embodiment introduces a machine learning model and calibrates image data based on 3D point cloud data to improve the accuracy of the resulting 3D inspection map, thereby enhancing the accuracy of inspections for military inspection equipment. Furthermore, 3D point cloud data is acquired only for obscured areas using LiDAR to reduce data acquisition complexity. A 3D inspection map is then generated through splicing to facilitate map updates, thereby improving inspection performance.

[0079] As a specific implementation method, Figure 4 As shown in the figure, handle the abnormal inspection situation:

[0080] Step S210: When it is determined that the abnormality corresponding to the target image data is the presence of an uncontrolled device or person in the target area, random disturbance is added to the target image data and sent to the background control end, and at the same time, it is broadcast to the uncontrolled device to make it leave the target area.

[0081] Step S212: When it is determined that the abnormality corresponding to the target image data is an abnormal operation of the equipment deployed in the target area, the abnormally operating equipment is continuously observed and inspected for a preset number of times; if each observation and inspection is determined to be an equipment abnormality, random disturbance is added to the target image data and then sent to the background control end.

[0082] At this time, based on steps S210 to S212, equipment abnormalities and personnel abnormalities can be handled differently to better ensure national defense.

[0083] At the same time, as a specific implementation method, Figure 5 As shown, set the inspection path to facilitate inspection:

[0084] Step S208.2: Obtain a list of abnormal devices in the previous time period, wherein the list of abnormal devices includes the abnormal device number, abnormal device location, and abnormal device frequency.

[0085] Step S208.4: planning an inspection route within the three-dimensional inspection map based on the abnormal equipment list; and performing inspection based on the inspection route.

[0086] At this time, based on steps S208.2 to S208.4, path planning is also used to prioritize inspections of abnormal equipment to ensure the normal operation of deployed equipment, thereby better ensuring national defense.

[0087] It can be seen that this embodiment further optimizes the method by setting paths that prioritize abnormal devices and handling abnormal situations differently.

[0088] The above method process can be executed in a processor or stored in a memory (or computer-readable medium). Computer-readable media includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmitting media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0089] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0090] Based on this, this embodiment also provides an inspection system for a military inspection device. Figure 6 As shown, the system includes:

[0091] The first acquisition module is used to acquire a number of corresponding first historical image data, second historical image data and historical three-dimensional point cloud data based on a historical database; wherein, the first historical image data is image data with occlusion in an abnormal environment, and the second historical image data is clear image data in a normal environment.

[0092] A model training module is used to iteratively train a machine learning calibration model using the first historical image data and the historical three-dimensional point cloud data as input until the error between the output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model.

[0093] The second acquisition module is used to obtain a number of measured image data of the target area based on the camera mounted on the aerial survey equipment, and at the same time obtain a number of measured three-dimensional point cloud data of the target area based on the laser radar mounted on the aerial survey equipment; and input the several measured image data and the several measured three-dimensional point cloud data into the target calibration model in sequence to obtain a number of target image data.

[0094] The actual inspection module is used to splice the multiple target image data to obtain a three-dimensional inspection map and load it into the military inspection device so that it can inspect the inspection area to obtain multiple target image data, and input the target image data into a target recognition model to determine whether there is an abnormality; wherein, the target recognition model is loaded on the military inspection device.

[0095] Since the system is used to execute the method, the above description will not be repeated here.

[0096] For example, the following modules are included to acquire the target recognition model:

[0097] The second acquisition module is used to acquire a number of historical image data to construct a training set and a verification set.

[0098] The training and verification module is used to train the initial recognition model based on the training set and to verify the trained initial recognition model based on the verification set.

[0099] An iterative updating module is used to repeatedly call the training verification module to iteratively train the initial recognition model until the target recognition model is obtained.

[0100] For example, in order to perform adaptive inspection of a military inspection device, the actual inspection module further includes:

[0101] The acquisition unit is used to acquire a list of abnormal devices in a forward-tracing time period, wherein the list of abnormal devices includes the abnormal device number, abnormal device location, and abnormal device frequency.

[0102] A planning unit is used to plan an inspection path within the three-dimensional inspection map based on the abnormal equipment list; and perform inspection based on the inspection path.

[0103] For example, the following models are included to perform follow-up processing when inspection anomalies occur:

[0104] The first judgment module is used to judge that when the abnormality corresponding to the target image data is the presence of uncontrolled equipment or personnel in the target area, random disturbance is added to the target image data and then sent to the background control end, and at the same time, broadcast to the uncontrolled equipment to make it leave the target area.

[0105] The second judgment module is used to judge that when the abnormality corresponding to the target image data is an abnormal operation of the equipment deployed in the target area, the equipment with abnormal operation is continuously observed and inspected for a preset number of times; if each observation and inspection is judged to be an equipment abnormality, random disturbance is added to the target image data and then sent to the background control end.

[0106] For example, in order to further improve the inspection accuracy, the system is configured to further include:

[0107] The third judgment module is used to send update information to the remote control terminal when it determines that the equipment deployment or geographical features of the inspection area are inconsistent with the corresponding area in the three-dimensional inspection map, so that it can obtain updated image data and updated three-dimensional point cloud data of the corresponding area based on the aerial survey equipment.

[0108] The local update module is used to obtain an updated local three-dimensional map according to the method based on the updated image data and the updated three-dimensional point cloud data.

[0109] A loading and updating module is used to update the three-dimensional inspection map based on the local three-dimensional map and then reload it into the military inspection device.

[0110] Since the system is built based on the method, it also has the advantage of high accuracy when conducting inspections of military inspection devices based on the system.

[0111] At the same time, this embodiment also provides an electronic device. Figure 7 As shown, the electronic device includes at least one processor, the processor is coupled to a memory, a computer program is stored in the memory, and the computer program is configured to execute the method when executed by the processor.

[0112] Furthermore, this embodiment also provides a computer-readable storage medium, on which a computer program is stored, and the computer program is used to execute the method described.

[0113] While the present invention has been disclosed above with reference to preferred embodiments, this is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A patrol inspection method for a military patrol inspection device, characterized in that: include: Acquiring a plurality of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data based on a historical database; wherein the first historical image data is image data with occlusion under abnormal conditions, the second historical image data is image data with clarity under normal conditions, and the historical three-dimensional point cloud data corresponds to the occluded areas in the first historical image data; Iteratively training a machine learning calibration model using the first historical image data and the historical three-dimensional point cloud data as input until an error between an output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model; Acquire a plurality of measured image data of the target area based on a camera mounted on the aerial survey equipment, and simultaneously acquire a plurality of measured three-dimensional point cloud data of the target area based on a laser radar mounted on the aerial survey equipment; and sequentially input the plurality of measured image data and the plurality of measured three-dimensional point cloud data into the target calibration model to acquire a plurality of target image data; The target image data are spliced ​​to obtain a three-dimensional inspection map and loaded into a military inspection device so that it can inspect the inspection area to obtain a number of target image data, and the target image data are input into a target recognition model to determine whether there is an abnormality; wherein, the target recognition model is loaded on the military inspection device.

2. The inspection method of the military inspection device according to claim 1, characterized in that: The training process of the target recognition model includes: Obtain some historical image data to construct training sets and validation sets; Training the initial recognition model based on the training set, and verifying the trained initial recognition model based on the verification set; Repeat the previous step to iteratively train the initial recognition model until the target recognition model is obtained.

3. The inspection method of the military inspection device according to claim 1, characterized in that: After inputting the target image data into a target recognition model to determine whether there is an abnormality; comprising: When it is determined that the abnormality corresponding to the target image data is the presence of an uncontrolled device or person in the target area, random disturbance is added to the target image data and then sent to the background control end, and at the same time, a broadcast is broadcast to the uncontrolled device to make it leave the target area; When it is determined that the abnormality corresponding to the target image data is an abnormal operation of the equipment deployed in the target area, the abnormally operating equipment is continuously observed and inspected for a preset number of times; if each observation and inspection is determined to be an equipment abnormality, random disturbance is added to the target image data and then sent to the background control end.

4. The inspection method of the military inspection device according to claim 1, characterized in that: The method of splicing the plurality of target image data to obtain a three-dimensional inspection map and loading the map into a military inspection device so that the device can inspect the inspection area to obtain a plurality of target image data includes: Obtain a list of abnormal devices within the previous time period, including the abnormal device number, abnormal device location, and abnormal device frequency; An inspection path is planned within the three-dimensional inspection map based on the abnormal equipment list; and an inspection is performed based on the inspection path.

5. The inspection method of the military inspection device according to claim 1, characterized in that: include: When it is determined that the equipment deployment or geographical features of the inspection area are inconsistent with the corresponding area in the three-dimensional inspection map, an update message is sent to the remote control terminal so that the remote control terminal can obtain updated image data and updated three-dimensional point cloud data of the corresponding area based on the aerial survey equipment; Acquire an updated local three-dimensional map based on the updated image data and the updated three-dimensional point cloud data according to the method; The three-dimensional inspection map is updated based on the local three-dimensional map and then reloaded into the military inspection device.

6. A patrol system for a military patrol device, characterized in that: include: A first acquisition module is configured to acquire a plurality of corresponding first historical image data, second historical image data, and historical three-dimensional point cloud data based on a historical database; wherein the first historical image data is image data with occlusion under abnormal conditions, the second historical image data is image data with clarity under normal conditions, and the historical three-dimensional point cloud data corresponds to the occluded areas in the first historical image data; a model training module, configured to iteratively train a machine learning calibration model using the first historical image data and the historical three-dimensional point cloud data as inputs until an error between an output and the corresponding second historical image data is less than a preset error threshold to obtain a target calibration model; The second acquisition module is configured to acquire a plurality of measured image data of the target area based on a camera mounted on the aerial survey equipment, and simultaneously acquire a plurality of measured three-dimensional point cloud data of the target area based on a laser radar mounted on the aerial survey equipment; and sequentially input the plurality of measured image data and the plurality of measured three-dimensional point cloud data into the target calibration model to acquire a plurality of target image data; The actual inspection module is used to splice the multiple target image data to obtain a three-dimensional inspection map and load it into the military inspection device so that it can inspect the inspection area to obtain multiple target image data, and input the target image data into a target recognition model to determine whether there is an abnormality; wherein, the target recognition model is loaded on the military inspection device.

7. The inspection system of the military inspection device according to claim 6, characterized in that: include: A first judgment module is configured to, when determining that the abnormality corresponding to the target image data is the presence of an uncontrolled device or person in the target area, add random perturbations to the target image data and send the result to the backend control terminal, while simultaneously broadcasting to the uncontrolled device to cause it to leave the target area; The second judgment module is configured to, when determining that the abnormality corresponding to the target image data is an abnormal operation of equipment deployed in the target area, conduct continuous observation and inspection of the abnormally operating equipment for a preset number of times; If each observation and inspection determines that the equipment is abnormal, random disturbance is added to the target image data and then sent to the background control end.

8. The inspection system of the military inspection device according to claim 6, characterized in that: The actual inspection module includes: An acquisition unit is used to acquire a list of abnormal devices in a previous time period, wherein the list of abnormal devices includes the abnormal device number, abnormal device location, and abnormal device frequency; A planning unit is used to plan an inspection path within the three-dimensional inspection map based on the abnormal equipment list; and perform inspection based on the inspection path.

9. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is coupled to a memory, wherein a computer program is stored in the memory, and wherein the computer program is configured to execute any one of the methods described in 1-5 when executed by the processor.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is used to execute the method described in any one of 1-5.

Images