Automatic detection system and automatic detection method for enclosed spaces

Abstract

This invention relates to an automatic detection system and method for enclosed spaces. The automatic detection system includes: an interactive device; a motion platform; an environmental sensing device; a defect detection device; a memory; and a processing device configured to process environmental data from the environmental sensing device to control the motion platform and the defect detection device, and to process detection data generated by the defect detection device to generate a detection report. The interactive device, environmental sensing device, defect detection device, memory, and processing device are mounted on the motion platform, and the interactive device can be operated to identify enclosed spaces and enable the automatic detection system to automatically perform detection in an automatic detection mode based on digital prototype data of the enclosed space. The automatic detection system and method for enclosed spaces according to the invention can automatically perform detection, reducing the workload of operators and improving detection efficiency.

Description

Technical Field

[0001] This invention relates to an automatic detection system and an automatic detection method, and more specifically, to an automatic detection system and method for detecting enclosed spaces (e.g., the interior space of an aircraft). Background Technology

[0002] The content in this section only provides background information related to this invention and may not constitute prior art.

[0003] Inspecting the condition of enclosed environments is common in many industrial sectors. For example, in the aircraft industry, as part of aircraft maintenance, the condition of the cargo hold is typically inspected before flight. For civil aircraft, the cargo hold, as part of the overall aircraft structure, is primarily used to carry passenger luggage and other cargo. During aircraft maintenance checks, it is necessary to inspect cargo hold facilities (e.g., cargo hold floors and side panels) for damage (e.g., dents or cracks) to facilitate timely repairs.

[0004] While robot-based automated inspection technologies are widely used in aircraft maintenance—for example, inspection robots can be used to move around an aircraft to inspect the appearance of its components—the specific spatial characteristics of an aircraft's interior, such as limited space and insufficient lighting, restrict the application of inspection robots typically used in open spaces, and also cannot guarantee the accuracy of the inspection results.

[0005] Therefore, inspections of aircraft interior spaces (e.g., cargo holds) are still typically conducted by personnel who enter the space according to a checklist. With technological advancements, the expectation is to gradually shift from manual on-site inspections by personnel to automated, robot-based inspections to improve efficiency.

[0006] Therefore, it is necessary to design a detection robot suitable for enclosed spaces. Summary of the Invention

[0007] One object of this invention is to provide an automatic detection system for enclosed spaces, reducing the workload of operators and improving detection efficiency. Another object of this invention is to improve the accuracy of the detection results from automatic detection.

[0008] One aspect of the present invention provides an automatic detection system for an enclosed space. The automatic detection system includes: an interactive device providing interaction between an operator and the automatic detection system; a motion platform capable of moving within the enclosed space; an environmental sensing device configured to acquire environmental data of the motion platform during its movement; a defect detection device configured to detect the enclosed space to generate detection data, the defect detection device including a detection imaging device; a memory storing digitized prototype data of the enclosed space; and a processing device communicating with the motion platform, the interactive device, the environmental sensing device, the defect detection device, and the memory, and configured to process the environmental data from the environmental sensing device to control the motion platform and the defect detection device, and to process the detection data generated by the defect detection device to generate a detection report. The interactive device, the environmental sensing device, the defect detection device, the memory, and the processing device are mounted on the motion platform, and the interactive device can be operated to identify the enclosed space and cause the automatic detection system to automatically perform detection in an automatic detection mode based on the digitized prototype data of the enclosed space.

[0009] Therefore, the automated inspection system utilizes digital prototype data from enclosed spaces (e.g., the interior of an aircraft, such as a cargo hold or passenger cabin) to automatically perform inspections and generate inspection reports, reducing the workload of operators and improving inspection efficiency.

[0010] The processing apparatus includes: a data acquisition and preprocessing unit configured to acquire digital prototype data of an enclosed space from a memory, acquire environmental data from an environmental sensing device and detection data from a defect detection device, and preprocess the environmental data and detection data; a control unit configured to communicate with the data acquisition and preprocessing unit to determine a target detection path; and a detection analysis unit configured to communicate with the data acquisition and preprocessing unit to analyze the detection results and generate a detection report. The processing apparatus is configured such that: in automatic detection mode, the data acquisition and preprocessing unit searches for available learning data in the memory, the learning data including the target detection path of the motion platform and detection marker point data; when the data acquisition and preprocessing unit finds available learning data, the control unit acquires the target detection path from the learning data; when the data acquisition and preprocessing unit does not find available learning data, the control unit automatically generates the target detection path based on the digital prototype data of the enclosed space acquired by the data acquisition and preprocessing unit; and the control unit controls the motion platform to move along the target detection path.

[0011] The automatic inspection system can automatically perform inspections based on existing learning data and digital prototype data from enclosed spaces (e.g., the interior of an aircraft), further improving inspection efficiency and accuracy.

[0012] The interactive device can also be operated to enable the automatic detection system to perform detection in a learning detection mode. In the learning detection mode, the operator operates the motion platform and the defect detection device, and inputs the detection marker point data via the interactive device. The processing unit is configured such that, in the learning detection mode, the control unit communicates with the data acquisition and preprocessing unit to automatically identify the motion path of the motion platform and the detection marker point data, and stores the identified motion path of the motion platform and the detection marker point data in the memory as learning data for the automatic detection mode.

[0013] Therefore, the automated inspection system can utilize the operator's experience and operational data, while also combining digital prototype data from enclosed spaces (e.g., the interior of an aircraft), further improving the accuracy of the inspection results.

[0014] The detection and analysis unit is configured to update the learning data for the automatic detection mode of enclosed spaces based on the detection results of the enclosed spaces.

[0015] Therefore, when performing automatic detection, the automatic detection system can further improve the detection process based on the experience data from previous detections, avoid missed detections, and ensure the accuracy of the detection results.

[0016] The environmental sensing device includes: a motion camera configured to capture positioning images as it moves with the motion platform; and an inertial measurement unit and a wheel odometer configured to measure the motion of the motion platform. The processing unit is configured such that a data acquisition and preprocessing unit preprocesses the positioning images captured by the motion camera and the measurement data from the inertial measurement unit and the wheel odometer, so that the control unit can determine the pose of the motion platform.

[0017] The automatic detection system measures the motion of the motion platform from multiple aspects using action cameras, inertial measurement units, and wheel odometers. After fusing the data from these sensors, the accurate pose of the motion platform can be obtained, thereby enabling accurate positioning and navigation.

[0018] The data acquisition and preprocessing unit is configured to perform texture optimization processing on the localization image. Therefore, the quality of the localization image can be improved, thereby enhancing the robustness of automatic detection.

[0019] The environmental sensing device also includes a light intensity sensor, and the automatic detection system also includes an illumination device mounted on a motion camera or defect detection device. The processing unit is configured to selectively activate the illumination device based on the detection results from the light intensity sensor.

[0020] Therefore, the automatic detection system can overcome the problem of insufficient lighting in enclosed spaces, ensure the quality of the captured images, and thus ensure the accuracy of navigation and positioning as well as the accuracy of detection and analysis.

[0021] The environmental sensing device further includes: an impact sensor for detecting collisions in the automatic detection system; and an obstacle detection sensor configured to detect obstacles on the motion path of the motion platform, the obstacle detection sensor including at least one of an ultrasonic sensor, a lidar sensor, and a time-of-flight sensor. The processing unit is configured such that a data acquisition and preprocessing unit acquires and preprocesses measurement data from the impact sensor and the obstacle detection sensor for use by the control unit to control the motion of the motion platform.

[0022] Therefore, the automatic detection system can detect obstacles, detect collisions or avoid collisions, and control the movement of the motion platform accordingly.

[0023] The automatic detection system also includes an outer cover, which is mounted on the motion platform to house at least partially the device mounted on the motion platform inside the cover, and the cover is fitted with anti-collision strips.

[0024] Therefore, the automatic detection system can have a better appearance and also provides protection for the devices mounted on the motion platform.

[0025] The motion platform is equipped with two drive wheels and multiple driven wheels, the multiple driven wheels being configured to be independently controllable.

[0026] The automatic detection system can be well supported, can move stably, and also improves the flexibility of the automatic detection system's movement.

[0027] An enclosed space is either the cargo hold or the passenger cabin of an aircraft.

[0028] Another aspect of the present invention provides an automatic detection method for enclosed spaces. The automatic detection method is executed by an automatic detection system, which includes an interactive device, a motion platform, an environmental sensing device, a defect detection device, a memory, and a processing device. The interactive device, environmental sensing device, defect detection device, memory, and processing device are mounted on the motion platform, and the processing device communicates with the interactive device, motion platform, environmental sensing device, defect detection device, and memory. The automatic detection method includes the following steps: operating the interactive device to identify and confirm the enclosed space; operating the interactive device to cause the automatic detection system to automatically perform detection in an automatic detection mode based on digital prototype data of the enclosed space; and automatically generating a detection report.

[0029] When performing detection in automatic detection mode, the automatic detection method includes: searching for digital prototype data of the enclosed space in the memory; searching for available learning data in the memory, the learning data including the target detection path of the motion platform and detection marker point data; when available learning data is found, obtaining the target detection path from the learning data; when no available learning data is found, automatically generating the target detection path based on the digital prototype data of the enclosed space; and controlling the motion platform to move along the target detection path and controlling the defect detection device to detect the enclosed space to generate detection data.

[0030] The automatic detection method further includes: operating an interactive device to enable the automatic detection system to perform detection in a learning detection mode; and wherein, when performing detection in the learning detection mode, the automatic detection method includes: an operator operating the automatic detection system to control the movement of a motion platform, controlling a defect detection device to generate detection data, and inputting detection marker point data via the interactive device; a processing device acquiring environmental data from an environmental sensing device, automatically identifying the motion path of the motion platform and the detection marker point data, and storing the identified motion path of the motion platform and the detection marker point data in a memory for use as learning data for the automatic detection mode.

[0031] The environmental sensing device includes an action camera, an inertial measurement unit (IMU), and a wheel odometer. The automatic detection method includes processing the positioning images captured by the action camera and the measurement data from the IMU and wheel odometer to calculate the real-time pose of the motion platform.

[0032] The environmental sensing device also includes an obstacle detection sensor and an impact sensor, wherein the obstacle detection sensor includes at least one of an acoustic sensor, a lidar sensor, and a time-of-flight sensor. When detection is performed in an automatic detection mode, the automatic detection method further includes controlling the movement of the motion platform based on the measurement results of the obstacle detection sensor and the impact sensor.

[0033] The environmental sensing device also includes a light intensity sensor, and the automatic detection system also includes an illumination device mounted on the motion camera or defect detection device. When performing detection in automatic detection mode, the automatic detection method further includes automatically activating the illumination device when the light intensity sensor detects insufficient light.

[0034] The defect detection device includes a detection imaging device. When performing detection in automatic detection mode, the automatic detection method includes: when the detection mark point is reached, controlling the speed of the motion platform and the pose of the detection imaging device based on learning data.

[0035] The automatic detection method also includes updating the learning data for the automatic detection mode of the enclosed space based on the detection results of the enclosed space.

[0036] The motion platform is equipped with two drive wheels and multiple driven wheels. When performing detection in automatic detection mode, the automatic detection method also includes raising some of the multiple driven wheels.

[0037] This invention provides an automatic detection system and method for enclosed spaces. The automatic detection system and method for enclosed spaces according to this invention can automatically perform detection and generate detection reports using digital prototype data of the enclosed space, significantly reducing the workload of operators and improving detection efficiency. Furthermore, the automatic detection system and method for enclosed spaces according to this invention can also utilize operator experience data to ensure the accuracy of the detection results. Attached Figure Description

[0038] Embodiments of the invention will now be described by way of example only with reference to the accompanying drawings. In the drawings, the same features or parts are indicated by the same reference numerals, and the drawings are not necessarily drawn to scale.

[0039] Figure 1 A block diagram illustrating an automatic detection system according to an embodiment of the present invention is shown;

[0040] Figure 2 A perspective view of an automatic detection system according to an example of the present invention is shown;

[0041] Figure 3 It shows that Figure 2 A 3D view of the automatic detection system after the outer cover has been removed;

[0042] Figure 4 A bottom view of the automatic detection system is shown, illustrating the status of each of the system's wheels.

[0043] Figure 5 A flowchart of an automatic detection method according to an embodiment of the present invention is shown;

[0044] Figure 6 A flowchart of the automatic detection mode of the automatic detection method according to the present invention is shown;

[0045] Figure 7 A schematic diagram illustrating a navigation operation according to an example of the present invention is shown; and

[0046] Figure 8 The diagram shows the path of the automatic detection system during detection. Detailed Implementation

[0047] The following description is exemplary in nature and is not intended to limit the invention, its application, or its uses. It should be understood that in all these figures, similar reference numerals indicate the same or similar parts and features. The figures are only schematic representations of the concept and principles of embodiments of the invention and do not necessarily show the specific dimensions and scale of each embodiment. Certain parts in specific figures may be exaggerated to illustrate relevant details or structures of embodiments of the invention.

[0048] Figure 1 A block diagram of an automatic detection system 1 according to an embodiment of the present invention is shown. Figure 1 As shown, the automatic detection system 1 includes a motion platform 10, an interaction device 20, an environmental sensing device 30, a defect detection device 40, a processing device 50, and a memory 60. The interaction device 20, the environmental sensing device 30, the defect detection device 40, the processing device 50, and the memory 60 can all be mounted on the motion platform 10.

[0049] The motion platform 10 can be equipped with a drive motor and multiple traveling wheels, enabling stable movement. The traveling wheels of the motion platform 10 may include drive wheels that drive the motion platform 10 to move and driven wheels that move accordingly as the drive wheels move.

[0050] The interactive device 20 provides an interface for interaction between the automatic detection system 1 and the operator. After the automatic detection system 1 is powered on, the operator can use the interactive device 20 to start or stop the operation of the automatic detection system 1, select the detection mode, and query the detection records.

[0051] The environmental sensing device 30 is configured to sense the environment in which the motion platform 10 is located during its movement, acquire environmental data of the motion platform 10, and determine the pose of the motion platform 10 and its mounted devices in real time for positioning and navigation. The environmental sensing device 30 includes various sensors or detectors for sensing the environment, such as, but not limited to, an action camera 31, an inertial measurement unit (IMU) 32, a wheel odometer 33, an impact sensor 34, a light intensity sensor 35, an ultrasonic sensor 36, a lidar 37, and a time-of-flight (TOF) sensor 38. The environmental sensing device 30 may also include other sensors or detectors. The action camera 31 can be mounted on the motion platform 10 such that the pose of the action camera 31 relative to the motion platform 10 is adjustable. In one example, the action camera 31 can be mounted on the motion platform 10 via a motion-controlled gimbal, so that the shooting pose of the action camera 31 can be changed by moving the gimbal. The action camera 31 is configured to capture environmental images as positioning images during its movement with the motion platform 10. Preferably, the action camera 31 is a high-speed binocular camera. The environmental sensing device 30 also includes an illumination device 39, which can be mounted on the action camera 31. In the event of insufficient light, for example when the light intensity sensor 35 detects insufficient light, the illumination device 39 can be automatically activated to illuminate the area, thereby improving the quality of the positioning image captured by the action camera 31 and thus improving positioning accuracy.

[0052] The IMU 32 and the wheel odometer 33 are configured to measure the motion, distance, and orientation of the motion platform 10 as it moves. The measurement data from the IMU 32 and the wheel odometer 33, along with the positioning images captured by the action camera 31, are processed to determine the pose of the motion platform 10 and the devices mounted on it.

[0053] Impact sensor 34 is used to detect collisions that occur to the motion platform 10 during its movement. Ultrasonic sensor 36, lidar 37, and TOF sensor 38 form the obstacle detection sensor of the environmental sensing device 30, used to detect obstacles that the motion platform 10 may encounter during its movement, thereby facilitating obstacle avoidance during the navigation of the motion platform 10. Lidar 37 can be a 2D or 3D lidar. It should be noted that the obstacle detection sensor of the environmental sensing device 30 may include at least one of ultrasonic sensor 36, lidar 37, and TOF sensor 38, or may include other sensors besides ultrasonic sensor 36, lidar 37, and TOF sensor 38.

[0054] The defect detection device 40 is configured to detect and measure defects in an object to be inspected (e.g., an enclosed space within an aircraft, such as a cargo hold or passenger cabin). The defect detection device 40 includes an image capture device 41 and a three-dimensional measurement device 42. The image capture device 41 is configured to capture images of the object to be inspected at predetermined time intervals as the motion platform 10 moves within the area of ​​the object to be inspected (e.g., an enclosed space within an aircraft, such as a cargo hold), and to store the captured images in a memory 60. The defect detection device 40 is mounted on the motion platform 10 such that the pose of the image capture device 41 relative to the mounting platform 10 is adjustable. The three-dimensional measurement device 42 is used to measure the three-dimensional dimensions of the defects.

[0055] The memory 60 is used to store the identification number of the object to be inspected and its corresponding digital prototype model (DMU) data, learning data, inspection rules, and inspection records.

[0056] The processing device 50 is configured to communicate with the motion platform 10, the interaction device 20, the environmental sensing device 30, the defect detection device 40, and the memory 60. The processing device 50 includes a data acquisition and preprocessing unit 51, a control unit 52, and a detection and analysis unit 53. The data acquisition and preprocessing unit 51 includes a detection object data acquisition module 511, a positioning data preprocessing module 512, a collision and obstacle avoidance preprocessing module 513, and a detection data preprocessing module 514. The detection object acquisition module 511 acquires user input from the interaction device 20 and, based on the user input (e.g., the identification number of the detection object), acquires the data of the detection object (e.g., the DMU data, learning data, detection rules, and detection records of the detection object, etc.) from the memory 60.

[0057] The positioning data preprocessing module 512 can acquire positioning image data captured by the motion camera 31 and measurement data from the IMU 32 and wheel odometer 33 from the environmental sensing device 30 and perform preprocessing. Combined with the data of the detected object acquired by the detection object data acquisition module 511, it calculates the real-time pose of the motion platform 10 and its mounted devices relative to the detected object. Positioning image preprocessing includes image processing such as feature point extraction from the positioning image acquired by the motion camera 31. Preferably, it also includes texture optimization processing of the positioning image before feature point extraction to improve the quality of the positioning image, thereby improving the robustness of navigation positioning calculations. Texture optimization processing of the positioning image includes image processing of the positioning image acquired by the motion camera 31 to extract image texture information. When the texture in the positioning image is determined to be sparse, for example, when the surface of the portion captured by the motion camera 31 is smooth or the texture of the positioning image is sparse due to reflection, texture is added to the positioning image to improve its quality, thereby improving the robustness of navigation positioning and automatic detection performed based on the navigation positioning.

[0058] The collision and obstacle avoidance preprocessing module 513 can acquire and preprocess measurement data from the impact sensor 34, ultrasonic sensor 36, lidar 37 and TOF sensor 38 from the environmental sensing device 30, so as to detect collisions and obstacles and perform collision avoidance and obstacle avoidance processing.

[0059] The detection data preprocessing module 514 acquires detection data (including detection images captured by the detection imaging device 41 and three-dimensional data measured by the three-dimensional measuring device 42) from the defect detection device 40, preprocesses the detection data, associates the detection data with the corresponding real-time pose, forms detection position metadata, and stores it in the memory 60.

[0060] The control unit 52 performs control based on the various data acquired by the data acquisition and preprocessing unit 51 and the preprocessing results of the various data, including the control of the motion platform 10 and the defect detection device 40. The control unit 52 includes a path generation module 521, a collision detection module 522, an obstacle detection module 523, and a control module 524. The control module 524 performs corresponding control based on the processing results of the path generation module 521, the collision detection module 522, and the obstacle detection module 523. The path generation module 521 generates the movement path of the automatic detection system 1 in real time based on the real-time pose determined by the positioning data preprocessing module 512 and the detection object data acquired by the detection object acquisition module 511. The control module 524 controls the movement of the motion platform 10, the pose of the motion camera 31, and the pose of the detection and shooting device 41 based on the generated movement path. The collision detection module 522 detects collisions based on the preprocessing of the data from the impact sensor 34 by the collision and obstacle avoidance preprocessing module 513. The obstacle detection module 523 detects obstacles based on the data processed by the collision and obstacle avoidance preprocessing module 513 from the obstacle detection sensors (ultrasonic sensor 36, lidar 37, and TOF sensor 38). Once a collision is detected or an obstacle is detected ahead, the path generation module 521 regenerates the movement path, and the control module 524 controls the motion platform 10 based on the generated new movement path to quickly eliminate the current collision or bypass the obstacle to avoid a collision.

[0061] The detection analysis unit 53 determines the detection result based on the real-time pose determined by the positioning data preprocessing module 512, the DMU data of the detection object acquired by the detection object acquisition module 511 from the memory 60, and the corresponding detection specifications of the detection object. It also determines the detection result based on the preprocessing of the detection image captured by the detection imaging device 41 by the detection data preprocessing module 514, and stores the detection result in the memory 60. The detection result includes the presence of defects (e.g., dents or cracks), the location, size (length, width, depth), and three-dimensional contour of the defects. When a defect is detected, the detection analysis unit 53 can activate the three-dimensional measurement device 42 to measure the defect area to obtain the three-dimensional size information of the defect. It should be noted that in this example, the detection analysis unit 53 is part of the processing device 50, and the three-dimensional measurement device 42 is part of the defect detection device 40. However, the invention is not limited to this; in other examples according to the invention, the detection analysis unit 53 and the three-dimensional measurement device 42 can be an integrated device.

[0062] Figure 2 A perspective view of an example of the automatic detection system 1 according to the present invention is shown. Figure 3 It shows Figure 2 The three-dimensional image of the automatic detection system 1 after removing the outer cover 70, and Figure 4 A bottom view of the automatic detection system 1 is shown. It includes an interaction device 20, an environmental sensing device 30, a defect detection device 40, a processing device 50, and a memory 60. Figure 2 and Figure 3 (Not shown in the figure) are all mounted on the motion platform 1 and housed within the outer casing 70, partially protruding from the outer casing 70. The outer casing 70 may be equipped with multiple anti-collision strips. In the example shown, the outer casing 70 is equipped with two anti-collision strips 71 and 72 spaced apart from each other, which reduces the impact on the devices inside the outer casing 70 even if a collision occurs with the object being detected, and also prevents the outer casing 70 from damaging the object it collides with. Furthermore, impact sensors 34 may be embedded within the anti-collision strips 71 and 72 of the outer casing 70.

[0063] In the example shown in the figure, the bottom of the motion platform 10 is equipped with multiple driving wheels, including two drive wheels 11 and 12 and four driven wheels 13-16, as shown in the figure. Figure 4As shown. Drive wheels 11 and 12 are connected to the output shaft of a drive motor (not shown), thereby driving the motion platform 10 to move under the drive of the drive motor. Drive wheels 11 and 12 are connected to the motion platform 10 via a support rod (not shown) with a damper (e.g., a damping spring) to reduce vibration. A steering linkage (not shown) is installed between drive wheels 11 and 12, which allows drive wheels 11 and 12 to rotate in the direction of travel. In addition, a differential device 17 is installed on the connecting shaft between drive wheels 11 and 12, allowing drive wheels 11 and 12 to move at different speeds to facilitate steering.

[0064] Driven wheels 13-16 can change their direction of movement according to the direction of movement of drive wheels 11 and 12. Driven wheels 13-16 are installed spaced apart from each other around the center of the bottom of the motion platform 10. In the example shown, driven wheels 13-16 are all-purpose wheels and are arranged at the four corners around the center of the bottom of the motion platform 10. Driven wheels 13-16 can be controlled to rise and fall independently, allowing some of the driven wheels to participate in the movement of the motion platform 10 as needed, thus providing better maneuverability. Preferably, the rising and falling of two driven wheels diagonally opposite each other in the driven wheels 13-16 are controlled simultaneously, thereby providing stable support for the motion platform 10. With the drive wheels 11 and 12 and driven wheels 13-16 arranged in this way, greater flexibility and stability can be provided for the movement of the motion platform 10.

[0065] The motion platform 10 is also equipped with a handle 18. When the outer cover 70 is installed, the handle 18 protrudes from a corresponding opening on the upper part of the outer cover. The operator can move the automatic detection system 1 by grasping the handle 18.

[0066] In the example shown in the figure, the interactive device 20 is mounted on the motion platform 10 and protrudes from a corresponding opening in the housing 70 for operator access. The interactive device 20 takes the form of an electronic display screen, such as a touch-sensitive electronic display screen. Other suitable forms are also possible for the interactive device 20.

[0067] In the example shown in the figure, the environmental sensing device 30 is mounted on the motion platform 10, and the motion camera 31 is a high-speed binocular camera. When installed in place, the impact sensor 34 is embedded in the anti-collision strips 71 and 72 of the outer casing 70, and the wheel odometer 33 is mounted on the drive wheel 11.

[0068] The automatic detection system 1 has two detection modes: learning detection mode and automatic detection mode.

[0069] The following will use the cargo hold of an aircraft as an example to introduce an automatic detection method using the automatic detection system 1 according to the present invention.

[0070] Figure 5 A flowchart of the automatic detection method according to the present invention is shown. Figure 5 As shown, in step S1, the automatic detection system 1 is moved to the vicinity of the aircraft's cargo door and powered on. The operator can power on the automatic detection system 1 using a power switch (not shown). Then, in step S2, the operator activates the automatic detection system 1 to identify and confirm the detected object by operating the interface of the interactive device 20. During this process, the motion camera 31 of the environmental sensing device 30 captures a picture of the QR code on the aircraft's cargo door. The data acquisition and preprocessing unit 51 of the processing device 50 acquires the QR code image captured by the motion camera 31, identifies the information in the QR code image, compares it with the data stored in the memory 60, identifies the information of the cargo hold to be detected, such as the aircraft type and number of the cargo hold, and displays the detected cargo hold information on the interface of the interactive device 20 for the operator to confirm.

[0071] After the operator confirms the information of the cargo hold to be inspected, in step S3, the operator selects the inspection mode on the interface of the interactive device 20. The operator can choose either the learning inspection mode or the automatic inspection mode. If the operator selects the learning inspection mode, the process proceeds to step S4, activating the learning inspection mode. In the learning inspection mode, the operator operates the automatic inspection system 1 to complete the inspection of the cargo hold. Specifically, in step S41, the operator and the automatic inspection system 1 enter the cargo hold to be inspected together. The operator controls the movement of the automatic inspection system 1 (the travel speed, travel path, dwell time at a certain point, etc. of the motion platform 10), and operates the defect detection device 40 to perform inspection, thereby executing the learning inspection. During the inspection in the learning inspection mode, the operator can input inspection marker point data through the interactive device 20. Specifically, the operator can record specific inspection locations as inspection marker points and input the inspection results through the interactive device 20. These specific inspection locations can be locations that are prone to defects (e.g., dents or cracks) based on the operator's experience, or locations where defects have already been detected. The processing device 50 processes the positioning data and other environmental measurement data acquired from the environmental sensing device 30, identifies the movement path of the motion platform 10 during this process, and identifies the detection markers recorded by the operator and the data related to the detection markers (e.g., the dwell time or movement speed of the motion platform 10, the shooting pose of the detection imaging device 41, etc.), and stores them in the memory 60 as learning data for the automatic detection mode. The identified movement path of the motion platform 10 can be used as the target detection path for the automatic detection mode. In step S42, the learning ends, and the operational data generated during this process is stored in the memory 60 as learning data for the automatic detection mode.

[0072] If the operator selects the automatic detection mode, the process proceeds to step S5 to prepare for automatic detection. Specifically, in step S51, the processing device 50 queries the memory 60 to confirm whether the learning data for the cargo hold is available. If the memory 60 contains the learning data for the cargo hold, in step S52, the target detection path for this detection is obtained, and in step S55, the automatic detection mode is entered to perform automatic detection. If the memory 60 does not contain the learning data for the cargo hold, in step S53, a message is displayed on the interactive device 20 indicating that the automatic detection system 1 does not currently have the learning data for the cargo hold, and the operator is asked to confirm whether to continue automatic detection. If the operator chooses not to perform automatic detection, a mode selection interface pops up on the interactive device 20, allowing the operator to reselect the learning detection mode or exit directly. If the operator confirms to continue automatic detection in step S53, in step S54, the control unit 52 of the processing device 50 will automatically generate the target detection path based on the DMU data and detection rules for the cargo hold obtained by the data acquisition and preprocessing unit 51. Then, proceed to step S55 to perform automatic detection.

[0073] After the inspection is completed in either the learning inspection mode or the automatic inspection mode, the automatic inspection system 1 returns to the starting point. In step S6, the inspection analysis unit 53 of the processing device 50 analyzes the inspection location metadata stored in the memory 60 based on the DMU data of the cargo hold, judges the inspection results according to the cargo hold inspection specifications, and generates an inspection report. Specifically, after the inspection is performed in the learning inspection mode, if the operator has input the inspection results through the interaction device 20 during the inspection process, the inspection analysis unit 53 can generate an inspection report based on the operator's input through the interaction device 20 and in conjunction with the DMU data of the cargo hold. The inspection report includes information such as whether a defect exists, the type of defect detected, the location of the defect, and the three-dimensional dimensions of the defect. In step S7, the areas where defects (e.g., pits or cracks) are detected are marked as new detection markers, and the data related to the new detection markers (the dwell time or speed of the motion platform 10, the shooting pose of the detection shooting device 41, etc.) are stored in the memory 60 to update the learning data of the cargo hold according to the detection results of the cargo hold for the next automatic detection.

[0074] In step S8, determine whether to continue the detection. If to continue, repeat steps S2 to S7. If not to continue, end the detection operation in step S9.

[0075] It should be noted that steps S6 and S7 can also be executed after the detection operation is completed, for example, after the automatic detection system 1 returns to the detection center, thereby reducing the real-time calculation load of the processing device 50.

[0076] Figure 6 It shows Figure 5 The automatic detection flowchart for step S55 is shown below. After automatic detection begins, in step S521, the detection object acquisition module 511 of the data acquisition and preprocessing unit 51 of the processing device 50 retrieves the DMU data and available learning data of the cargo hold from the memory 60. Using the DMU data, during movement, the cargo hold entrance, the cargo hold outline, and facilities within the cargo hold (e.g., floor locks, vents, etc.) can be identified, thereby enabling appropriate avoidance actions during movement, and determining the minimum safe distance during movement based on the identified cargo hold outline.

[0077] In step S522, the data acquisition and preprocessing unit 51 acquires and preprocesses the positioning image captured by the motion camera 31, and determines the current position of the automatic detection system 1 relative to the cargo hold by combining it with the DMU data of the cargo hold, thereby determining the detection starting point. In step S523, based on the determined detection starting point, and according to the target detection path of this automatic detection, and in conjunction with the DMU data of the cargo hold, the automatic detection system 1 is navigated and positioned, and the movement speed of the motion platform 10, the pose of the motion platform 10 and its mounted devices are controlled, so that the automatic detection system 1 moves along the target detection path and performs detection. During the navigation and positioning process, the positioning data preprocessing module 512 of the processing device 50 performs texture optimization processing on the positioning image captured by the motion camera 31 to improve the quality of the positioning image, thereby improving the robustness of navigation and positioning.

[0078] During the navigation motion along the target detection path, as shown in step C1, it is determined whether a detection marker point has been reached. If a detection marker point has been reached, in step C11, based on the learning data corresponding to the detection marker point stored in the memory 60, the pose of the motion platform 10 and its mounted motion camera 31 and detection and shooting device 41 are controlled, and the three-dimensional measuring device 42 measures the detection marker point.

[0079] During navigation, obstacles can be automatically identified. As shown in step C2, the obstacle detection module 523 of the control unit 52 determines whether an obstacle has been detected. After confirming that an obstacle has been detected, in step C21, based on the measurement data from the ultrasonic sensor 36, the lidar 37, and the TOF sensor 38, the obstacle detection module 523 of the control unit 52 determines the three-dimensional dimensions, position, etc., of the obstacle and compares them with the DMU data of the cargo hold to determine whether the obstacle is an inherent facility within the cargo hold (e.g., a floor lock). If it is determined that the obstacle is not an inherent facility within the cargo hold, in step C22, the obstacle is identified as an anomaly, and its information is stored in the memory 60.

[0080] Next, based on the information of the identified obstacle, step C23 determines whether the obstacle can be directly crossed. If it is determined that the obstacle can be crossed, in step C24, the control unit 52 controls the motion platform 10 to continue moving along the predetermined target detection path without changing its direction of motion, thus crossing the obstacle. During this process, a pair of driven wheels located diagonally among the driving wheels 13-16 can be raised to improve the passability of the motion platform 10 while ensuring its stability. For example, […]. Figure 4 Driven wheels 13 and 15 rise, allowing the motion platform 10 to move via drive wheels 11 and 13 and another pair of driven wheels 14 and 16. When it is determined that an obstacle cannot be crossed, in step C25, the motion platform 10 is controlled to detour around the obstacle to avoid a collision. For example, if the height of the obstacle exceeds 5 cm and the slope exceeds 10 degrees, it is considered that the obstacle cannot be crossed directly and therefore detour is necessary. During the detour, a suitable detour plan can be selected by combining the position of the motion platform 10 and the obstacle relative to the cargo hold. For example, the drive wheels 11 and 12 of the motion platform 10... Figure 4 The arrow K1 in the diagram indicates a turning point or, as shown in the diagram, a turning point. Figure 4 As indicated by arrow K3, they retract, and driven wheels 13-16 turn accordingly as indicated by arrow K2 or arrow K4. During this process, a pair of driven wheels located diagonally among driven wheels 13-16 may also rise. For example, when drive wheels 11, 12... Figure 4 When the direction indicated by arrow K1 is turned, driven wheels 13 and 15 can be raised.

[0081] During navigation, collisions can be detected and quickly mitigated. In step C3, once the impact sensor 34 of the environmental sensing device 30 detects a collision, the control unit 52 controls the motion platform 10 to apply emergency braking to the drive wheels 11 and 12 and control them to circumvent the collision, thereby quickly mitigating the impact. Additionally, in the automatic detection system 1, anti-collision strips 71 and 72 (such as...) are installed on the outer casing 70. Figure 2 (Best visible in the middle). Even in the event of a collision, on the one hand, the impact of the collision on the devices mounted on the motion platform 10 can be reduced, and on the other hand, the cargo compartment being inspected will not be damaged.

[0082] In step S524, it is determined whether the detection endpoint has been reached. If the detection endpoint has not been reached, navigation continues and detection and image capture are performed. If the detection endpoint has been reached, the automatic detection ends in step S525.

[0083] During navigation, the data acquisition and preprocessing unit 51 of the processing device 50 fuses data from multiple sensors and uses a simultaneous localization and mapping (SLAM) positioning scheme to determine the pose of the motion platform 10 and its mounted device in real time, and the control unit 52 controls the motion platform 10 and its mounted device accordingly.

[0084] Figure 7 A schematic block diagram of a SLAM-based localization scheme based on multi-sensor data fusion according to an example of the present invention is shown. In the example shown, the front-end processing of the SLAM-based localization scheme employs a feature point method. Figure 7 As shown, firstly, the data from each sensor is preprocessed. Automatic texture optimization is performed on the positioning images captured by the action camera 31 (binocular camera), and feature points are extracted from the optimized positioning image data. IMU data measured by the IMU 32 is preprocessed, and errors are calculated. Dead reckoning is performed on the data from the wheel odometer 33. The positioning image data from the action camera 31 is fused with the IMU data, and the IMU data is fused with the data from the wheel odometer 33. Based on this, positioning and mapping are then performed. Specifically, based on the fusion of multi-sensor data, feature point statistics and matching are performed, and loop closure detection is executed. After backend optimization processing, the real-time pose of the platform 10 is obtained. Furthermore, in the positioning and mapping processing, repositioning is performed based on obstacle information identified from LiDAR data, TOF sensor data, and ultrasonic data to achieve obstacle avoidance. In the above positioning scheme, by fusing multi-sensor data and performing loop closure detection, accumulated errors can be eliminated, pose drift can be minimized, and accurate navigation and positioning can be achieved. The pose of the device mounted on the motion platform 10 can be calculated based on the relative position between the device and the motion platform 10. During the detection process, the calculated real-time pose of the detection imaging device 41 is associated with the detection image captured by the detection imaging device 41 to form detection position metadata, which is then stored in the memory 60.

[0085] Figure 8 A schematic diagram of the automatic detection system 1 being used in cargo hold V is shown. Figure 8 The dashed line in the diagram shows the detection field of view of the automatic detection system 1.

[0086] The automatic detection system 1 and automatic detection method according to the present invention can flexibly select the detection mode as needed, and can learn from the detection mode learned from manual operation, combining the operator's experience and operation data into the automatic detection mode, thus effectively performing automatic detection and ensuring the accuracy of the detection results. Once in automatic detection mode, the automatic detection system 1 and automatic detection method can automatically perform detection without further remote operation control by the operator. During the detection process, the automatic detection system 1 and automatic detection method according to the present invention can accurately perform real-time positioning and navigation based on the measurement data of multiple sensors and combined with the DMU data of the object being detected, improving the accuracy of positioning and navigation, thereby further improving the accuracy of the detection results. Furthermore, the automatic detection system 1 and automatic detection method can detect obstacles on the movement path and avoid them, reducing or avoiding collisions. In addition, the outer casing 70 of the automatic detection system 1 is equipped with anti-collision strips 71 and 72, which can reduce the impact of collisions even in the event of a collision, reducing adverse effects on the various devices of the automatic detection system 1, and also preventing damage to the object being detected by collisions.

[0087] The automatic detection system 1 and automatic detection method according to the present invention have been described above with reference to the accompanying drawings, and the application of the automatic detection system and automatic detection method according to the present invention is illustrated by taking an aircraft cargo hold as the detection object. However, the above examples should not be construed as limiting the automatic detection system and automatic detection method according to the present invention. The automatic detection system and automatic detection method according to the present invention can also be applied to other application scenarios (e.g., ship cargo holds).

[0088] Exemplary embodiments of the present invention have been described in detail herein; however, it should be understood that the present invention is not limited to the specific embodiments described and shown above. Various modifications and variations can be made to the present invention by those skilled in the art without departing from its spirit and scope. All such modifications and variations fall within the scope of the present invention. Moreover, all components described herein can be replaced by other technically equivalent components.

Claims

1. An automatic detection system for enclosed spaces, the automatic detection system comprising: An interactive device that provides interaction between the operator and the automatic detection system; A motion platform, which is capable of moving within the enclosed space; An environmental sensing device is configured to acquire environmental data of the motion platform during its movement. A defect detection device, configured to detect the enclosed space to generate detection data, the defect detection device including a detection imaging device; A memory, wherein the memory stores digital prototype data of the enclosed space; as well as The processing device communicates with the motion platform, the interaction device, the environmental sensing device, the defect detection device, and the memory, and is configured to process environmental data from the environmental sensing device to control the motion platform and the defect detection device, and to process detection data generated by the defect detection device to generate a detection report. The interactive device, the environmental sensing device, the defect detection device, the memory, and the processing device are mounted on the motion platform, and the interactive device can be operated to identify the enclosed space and cause the automatic detection system to automatically perform detection in automatic detection mode based on the digital prototype data of the enclosed space. The processing device includes: The data acquisition and preprocessing unit is configured to acquire digital prototype data of the enclosed space from the memory, acquire environmental data of the environmental sensing device and detection data of the defect detection device, and preprocess the environmental data and the detection data. A control unit, configured to communicate with the data acquisition and preprocessing unit, is used to determine a target detection path; and The detection and analysis unit is configured to communicate with the data acquisition and preprocessing unit to analyze the detection results and generate a detection report. The processing device is configured such that, in the automatic detection mode, The data acquisition and preprocessing unit searches for available learning data in the memory, the learning data including the target detection path and detection marker data of the motion platform; When the data acquisition and preprocessing unit finds available learning data, the control unit acquires a target detection path from the learning data; when the data acquisition and preprocessing unit does not find available learning data, the control unit automatically generates a target detection path based on the digitized prototype data of the enclosed space acquired by the data acquisition and preprocessing unit. The control unit controls the motion platform to move along the target detection path.

2. The automatic detection system for enclosed spaces according to claim 1, wherein, The interactive device can also be operated to enable the automatic detection system to perform detection in a learning detection mode, in which an operator operates the motion platform and the defect detection device, and inputs detection marker point data via the interactive device; as well as The processing device is configured such that, in the learning and detection mode, the control unit communicates with the data acquisition and preprocessing unit to automatically identify the motion path and detection marker data of the motion platform, and stores the identified motion path and detection marker data of the motion platform in the memory as learning data for the automatic detection mode.

3. The automatic detection system for enclosed spaces according to claim 1, wherein, The detection and analysis unit is configured to update the learning data for the automatic detection mode of the enclosed space based on the detection results of the enclosed space.

4. The automatic detection system for enclosed spaces according to claim 1, wherein, The environmental sensing device includes: An action camera, configured to capture positioning images while moving with the motion platform; and An inertial measurement unit and a wheel odometer, the inertial measurement unit and the wheel odometer being configured to measure the motion of the motion platform. The processing device is configured such that the data acquisition and preprocessing unit preprocesses the positioning image captured by the action camera and the measurement data from the inertial measurement unit and the wheel odometer, so that the control unit can determine the pose of the motion platform.

5. The automatic detection system for enclosed spaces according to claim 4, wherein, The data acquisition and preprocessing unit is configured to perform texture optimization processing on the positioning image.

6. The automatic detection system for enclosed spaces according to claim 4, wherein, The environmental sensing device further includes a light intensity sensor, and the automatic detection system further includes an illumination device, which is mounted on the motion camera or the defect detection device; and The processing device is configured to selectively activate the lighting device based on the detection results of the light intensity sensor.

7. The automatic detection system for enclosed spaces according to claim 4, wherein, The environmental sensing device also includes: Impact sensor, used to detect collisions in the automatic detection system; and An obstacle detection sensor, configured to detect obstacles in the motion path of the motion platform, the obstacle detection sensor including at least one of an ultrasonic sensor, a lidar sensor, and a time-of-flight sensor; and The processing device is configured such that the data acquisition and preprocessing unit acquires and preprocesses the measurement data of the impact sensor and the obstacle detection sensor, so that the control unit can control the movement of the motion platform.

8. The automatic detection system for enclosed spaces according to any one of claims 1-7, wherein, The automatic detection system also includes an outer cover, which is installed on the motion platform to house at least partially the device mounted on the motion platform inside the outer cover, and the outer cover is equipped with anti-collision strips.

9. The automatic detection system for enclosed spaces according to any one of claims 1-7, wherein, The motion platform is equipped with two drive wheels and multiple driven wheels, the multiple driven wheels being configured to be independently controllable.

10. The automatic detection system for enclosed spaces according to any one of claims 1-7, wherein, The enclosed space is the cargo hold or passenger cabin of the aircraft.

11. An automatic detection method for enclosed spaces, the automatic detection method being executed by an automatic detection system, the automatic detection system comprising an interactive device, a motion platform, an environmental sensing device, a defect detection device, a memory, and a processing device, wherein, The interactive device, the environmental sensing device, the defect detection device, the memory, and the processing device are mounted on the motion platform, and the processing device communicates with the interactive device, the motion platform, the environmental sensing device, the defect detection device, and the memory. The automatic detection method is characterized by comprising the following steps: Operate the interactive device to identify and confirm the enclosed space; and Operate the interactive device to cause the automatic detection system to automatically perform detection in automatic detection mode based on the digital prototype data of the enclosed space; and Automatically generate test reports. When performing detection in the automatic detection mode, the automatic detection method includes: Search the memory for digital prototype data of the enclosed space; Search the memory for available learning data, which includes the target detection path and detection marker data of the motion platform; When available learning data is found, an object detection path is obtained from the learning data; when no available learning data is found, an object detection path is automatically generated based on the digitized prototype data of the enclosed space. The motion platform is controlled to move along the target detection path, and the defect detection device is controlled to detect the enclosed space to generate detection data.

12. The automatic detection method for enclosed spaces according to claim 11, wherein, The automatic detection method further includes: operating the interactive device to cause the automatic detection system to perform detection in a learned detection mode; and When performing detection using the learning detection mode, the automatic detection method includes: The operator operates the automatic detection system to control the movement of the motion platform, controls the defect detection device to generate detection data, and inputs the detection marker point data through the interactive device; The processing device acquires environmental data from the environmental sensing device, automatically identifies the motion path and detection marker data of the motion platform, and stores the identified motion path and detection marker data of the motion platform in the memory as learning data for the automatic detection mode.

13. The automatic detection method for enclosed spaces according to claim 11, wherein, The environmental sensing device includes a motion camera, an inertial measurement unit, and a wheel odometer. The automatic detection method includes: processing the positioning image captured by the action camera and the measurement data from the inertial measurement unit and the wheel odometer to calculate the real-time pose of the motion platform.

14. The automatic detection method for enclosed spaces according to claim 13, wherein, The environmental sensing device further includes an obstacle detection sensor and an impact sensor, wherein the obstacle detection sensor includes at least one of an acoustic sensor, a lidar, and a time-of-flight sensor; as well as When performing detection in the automatic detection mode, the automatic detection method further includes: controlling the movement of the motion platform based on the measurement results of the obstacle detection sensor and the impact sensor.

15. The automatic detection method for enclosed spaces according to claim 13, wherein, The environmental sensing device also includes a light intensity sensor, and the automatic detection system also includes an illumination device, which is mounted on the motion camera or the defect detection device. When performing detection in the automatic detection mode, the automatic detection method further includes: automatically activating the lighting device when the light intensity sensor detects insufficient light.

16. The automatic detection method for enclosed spaces according to claim 11, wherein, The defect detection device includes a detection and imaging device. When the detection is performed in the automatic detection mode, the automatic detection method includes: when the detection marker point is reached, controlling the speed of the motion platform and the pose of the detection and imaging device according to the learning data.

17. The automatic detection method for enclosed spaces according to any one of claims 11-16, wherein, The automatic detection method further includes updating the learning data of the automatic detection mode for the enclosed space based on the detection results of the enclosed space.

18. The automatic detection method for enclosed spaces according to any one of claims 11-16, wherein, The motion platform is equipped with two drive wheels and multiple driven wheels; When performing the detection in the automatic detection mode, the automatic detection method further includes raising some of the plurality of driven wheels.

Images