Automatic detection system and automatic detection method for closed space
By applying an automatic detection system in the internal space of the aircraft, using digital prototype data and multiple sensors, automatic detection of closed space is achieved, and the problem of low detection efficiency and accuracy in the prior art is solved, and detection efficiency and accuracy are improved and overcome the challenges of limited space and insufficient lighting.
Patent Information
- Application Number
- CN202310656217.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-05
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-06-05
AI Technical Summary
In the detection of the internal space of the aircraft, existing detection robots are difficult to ensure the accuracy and efficiency of the detection results due to limited space and insufficient lighting, and rely on manual detection efficiency.
An automatic detection system is designed, including interactive devices, motion platforms, environmental perception devices, defect detection devices, memory and processing devices. Using digital prototype data and sensor data, the automatic detection mode and learning detection mode can be used to realize the closed space. Automate inspection and report generation to ensure the accuracy and efficiency of inspection results.
It significantly reduces the workload of operators, improves detection efficiency and accuracy, and can accurately navigate and detect in closed spaces, overcomes insufficient lighting problems, and avoids collisions and obstacles, ensuring the reliability of detection results.
Smart Images

Figure CN119086584A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an automatic detection system and an automatic detection method, and more particularly, to an automatic detection system and an automatic detection method for detecting a closed space (such as an interior space of an aircraft). Background Art
[0002] The contents in this section merely provide background information related to the present invention and may not constitute prior art.
[0003] Testing the condition of a closed environment is common in many industrial fields. For example, in the field of aircraft, as part of the aircraft maintenance work, the condition of the aircraft's cargo hold is usually tested before flight. For civil aircraft, the cargo hold is part of the aircraft and is mainly used to load passenger luggage and other cargo. During the maintenance inspection of the aircraft, it is necessary to check whether the facilities in the cargo hold (for example, the cargo hold floor and side panels) are damaged (for example, dents or cracks) so that timely repairs can be made.
[0004] Although robot-based automatic inspection technology has been widely used in aircraft maintenance, for example, inspection robots can be used to move around an aircraft to inspect the appearance of aircraft components. However, due to its specific spatial characteristics, such as limited space and insufficient lighting, the application of inspection robots that are usually used in open spaces is limited in the interior space of an aircraft, and the accuracy of the inspection results cannot be guaranteed.
[0005] Therefore, when inspecting the interior space of an aircraft (e.g., a cargo hold), a staff member usually enters the interior space to inspect it according to a checklist. With the development of technology, it is expected that the inspection of the interior space of an aircraft will gradually change from manual inspection performed by staff on site to automatic inspection based on robots to improve efficiency.
[0006] Therefore, it is necessary to design a detection robot suitable for enclosed spaces. Summary of the invention
[0007] One object of the present invention is to provide an automatic detection system for closed spaces, which reduces the workload of operators and improves detection efficiency. Another object of the present invention is to improve the accuracy of the detection results of automatic detection.
[0008] One aspect of the present invention is to provide an automatic detection system for a closed space. The automatic detection system includes: an interactive device, which provides interaction between an operator and the automatic detection system; a motion platform, which can move in the closed space; an environmental sensing device, which is configured to obtain environmental data of the motion platform during the movement of the motion platform; a defect detection device, which is configured to detect the closed space to generate detection data, and the defect detection device includes a detection shooting device; a memory, which stores digital prototype data of the closed space; and a processing device, which communicates with the motion platform, the interactive device, the environmental sensing device, the defect detection device and the memory, and is configured to process the environmental data of the environmental sensing device to control the motion platform and the defect detection device, and to process the detection data generated by the defect detection device to generate a detection report. The interactive device, the environmental sensing device, the defect detection device, the memory and the processing device are mounted on the motion platform, and the interactive device can be operated to identify the closed space and enable the automatic detection system to automatically perform detection in an automatic detection mode according to the digital prototype data of the closed space.
[0009] Therefore, the automatic inspection system utilizes digital prototype data of a closed space (e.g., the interior space of an aircraft, such as a cargo hold or a passenger cabin) to automatically perform inspections and automatically generate inspection reports, thereby reducing the workload of operators and improving inspection efficiency.
[0010] The processing device includes: a data acquisition and preprocessing unit, which is configured to acquire digital prototype data of a closed space from a memory, acquire environmental data of an environmental sensing device and detection data of a defect detection device, and preprocess the environmental data and the detection data; a control unit, which is configured to communicate with the data acquisition and preprocessing unit to determine a target detection path; and a detection analysis unit, which is configured to communicate with the data acquisition and preprocessing unit to analyze the detection results and generate a detection report. The processing device is configured to: in an automatic detection mode, the data acquisition and preprocessing unit searches for available learning data in the memory, and the learning data includes a target detection path and detection mark point data of a motion platform; when the data acquisition and preprocessing unit searches for available learning data, the control unit acquires the target detection path from the learning data; when the data acquisition and preprocessing unit does not search for available learning data, the control unit automatically generates a target detection path according to the digital prototype data of the closed space acquired by the data acquisition and preprocessing unit; and the control unit controls the motion platform to move along the target detection path.
[0011] The automatic detection system can automatically perform detection based on existing learning data combined with digital prototype data of enclosed spaces (e.g., the interior space of an aircraft), further improving detection efficiency and accuracy.
[0012] The interactive device can also be operated to make the automatic detection system perform detection in a learning detection mode, in which the operator operates the motion platform and the defect detection device, and inputs the detection mark point data through the interactive device. The processing device is configured such that in the learning detection mode, the control unit communicates with the data acquisition and preprocessing unit to automatically identify the motion path of the motion platform and the detection mark point data, and store the identified motion path of the motion platform and the detection mark point data in the memory as learning data for the automatic detection mode.
[0013] Therefore, the automatic inspection system can utilize the operator's experience operation data and combine it with the digital prototype data of the enclosed space (eg, the interior space of the aircraft), thereby further improving the accuracy of the inspection results.
[0014] The detection analysis unit is configured to update the learning data for the automatic detection mode of the closed space according to the detection result of the closed space.
[0015] Therefore, when performing automatic detection, the automatic detection system can further improve the detection process based on the experience data of previous detections, avoid missed detections, and ensure the accuracy of the detection results.
[0016] The environment perception device includes: a motion camera, which is configured to capture a positioning image when the motion platform moves; and an inertial measurement unit and a wheel odometer, which are configured to measure the motion of the motion platform. The processing device is configured such that a data acquisition and preprocessing unit preprocesses the positioning image captured by the motion camera and the measurement data of the inertial measurement unit and the wheel odometer, so as to provide the control unit with the position and posture of the motion platform.
[0017] The automatic detection system measures multiple aspects of the motion platform's movement through motion cameras, inertial measurement units, and wheel odometers. After fusing the data from these sensors, the accurate position and posture of the motion platform can be obtained, thereby achieving accurate positioning and navigation.
[0018] The data acquisition and preprocessing unit is configured to perform texture optimization processing on the positioning image, thereby improving the quality of the positioning image and thus improving the robustness of the automatic detection.
[0019] The environment sensing device further includes a light intensity sensor, and the automatic detection system further includes a lighting device, which is arranged on the motion camera or the defect detection device. The processing device is configured to selectively start the lighting device according to the detection result of the light intensity sensor.
[0020] Therefore, the automatic detection system can overcome the problem of insufficient lighting in closed spaces, ensure the quality of the captured images, and further ensure the accuracy of navigation positioning and detection analysis.
[0021] The environment sensing device further includes: an impact sensor, which is used to detect a collision of the automatic detection system; and an obstacle detection sensor, which is configured to detect an obstacle on the motion path of the motion platform, and the obstacle detection sensor includes at least one of an ultrasonic sensor, a laser radar, and a time-of-flight sensor. The processing device is configured such that: the data acquisition and preprocessing unit acquires the measurement data of the impact sensor and the obstacle detection sensor and performs preprocessing for the control unit to control the motion of the motion platform.
[0022] Therefore, the automatic detection system can detect obstacles, detect collisions or avoid collisions, and control the movement of the motion platform accordingly.
[0023] The automatic detection system further comprises an outer cover which is mounted on the motion platform so as to at least partially accommodate the device carried by the motion platform inside the outer cover, and an anti-collision strip is mounted on the outer cover.
[0024] Thus, the automatic detection system can have a better appearance and also provide protection for the devices carried by the motion platform.
[0025] The motion platform is equipped with two driving wheels and a plurality of driven wheels, and the plurality of driven wheels are configured to be independently controllable.
[0026] The automatic detection system can be better supported, can move stably, and the flexibility of movement of the automatic detection system is also improved.
[0027] The enclosed space is the cargo hold or passenger cabin of the aircraft.
[0028] Another aspect of the present invention is to provide an automatic detection method for an enclosed space. The automatic detection method is performed by an automatic detection system, which includes an interactive device, a motion platform, an environmental sensing device, a defect detection device, a memory, and a processing device, wherein the interactive device, the environmental sensing device, the defect detection device, the memory, and the processing device are mounted on the motion platform, and the processing device communicates with the interactive device, the motion platform, the environmental sensing device, the defect detection device, and the memory. The automatic detection method includes the following steps: operating the interactive device to identify and confirm the enclosed space; and operating the interactive device so that the automatic detection system automatically performs detection in an automatic detection mode according to the digital prototype data of the enclosed space; and automatically generating a detection report.
[0029] When the inspection is performed in the automatic inspection mode, the automatic inspection method includes: searching the memory for digital prototype data of the closed space; searching the memory for available learning data, the learning data including the target detection path and detection mark point data of the motion platform; when the available learning data is searched, obtaining the target detection path from the learning data, and when the available learning data is not searched, automatically generating the target detection path according to the digital prototype data of the closed space; and controlling the motion platform to move along the target detection path, and controlling the defect detection device to inspect the closed space to generate inspection data.
[0030] The automatic detection method also includes: operating the interactive device so that the automatic detection system performs detection in a learning detection mode; and wherein, when the detection is performed in the learning detection mode, the automatic detection method includes: the operator operates the automatic detection system to control the movement of the motion platform, controls the defect detection device to generate detection data, and inputs the detection mark point data through the interactive device; the processing device obtains the environmental data of the environmental sensing device, and automatically identifies the motion path of the motion platform and the detection mark point data, and stores the identified motion path of the motion platform and the detection mark point data in the memory for use as learning data for the automatic detection mode.
[0031] The environment perception device includes a motion camera, an inertial measurement unit and a wheel odometer. The automatic detection method includes: processing the positioning image taken by the motion camera and the measurement data of the inertial measurement unit and the wheel odometer to calculate the real-time position and posture of the motion platform.
[0032] The environment sensing device further includes an obstacle detection sensor and an impact sensor, and the obstacle detection sensor includes at least one of an acoustic sensor, a laser radar, and a time-of-flight sensor. When the detection is performed in the automatic detection mode, the automatic detection method further includes: controlling the movement of the motion platform according to the measurement results of the obstacle detection sensor and the impact sensor.
[0033] The environment sensing device further includes a light intensity sensor, and the automatic detection system further includes a lighting device, which is arranged on the motion camera or the defect detection device. When the detection is performed in the automatic detection mode, the automatic detection method further includes: when the light intensity sensor detects insufficient light, automatically starting the lighting device.
[0034] The defect detection device includes a detection camera. When the detection is performed in an automatic detection mode, the automatic detection method includes: when reaching the detection mark point, controlling the speed of the motion platform and the posture of the detection camera according to the learning data.
[0035] The automatic detection method further includes: updating learning data for the automatic detection mode of the closed space according to the detection result of the closed space.
[0036] The motion platform is equipped with two driving wheels and a plurality of driven wheels. When performing detection in the automatic detection mode, the automatic detection method further comprises raising some of the plurality of driven wheels.
[0037] The present invention provides an automatic detection system and an automatic detection method for a closed space. The automatic detection system and the automatic detection method for a closed space according to the present invention can use the digital prototype data of the closed space to automatically perform detection and generate a detection report, which significantly reduces the workload of the operator and improves the detection efficiency. In addition, the automatic detection system and the automatic detection method for a closed space according to the present invention can also use the operator's experience operation data to ensure the accuracy of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Embodiments of the present invention will be described below by way of example only with reference to the accompanying drawings. In the accompanying drawings, the same features or components are indicated by the same reference numerals, and the drawings are not necessarily drawn to scale, and in the accompanying drawings:
[0039] Figure 1 shows a block diagram illustrating an automatic detection system according to one embodiment of the present invention;
[0040] Figure 2 A perspective view of an automatic detection system according to an example of the present invention is shown;
[0041] Figure 3 Shows the Figure 2 A three-dimensional image of the automatic detection system after the outer cover is removed;
[0042] Figure 4 A bottom view of the automatic detection system is shown, showing the status of each driving wheel of the automatic detection system;
[0043] Figure 5 A flow chart of an automatic detection method according to an embodiment of the present invention is shown;
[0044] Figure 6 A detection flow chart of an automatic detection mode of an automatic detection method according to the present invention is shown;
[0045] Figure 7 A schematic diagram showing a navigation operation according to an example of the present invention; and
[0046] Figure 8 A schematic diagram of the path of the automatic detection system when performing detection is shown. DETAILED DESCRIPTION
[0047] The following description is merely exemplary in nature and is not intended to limit the present invention, its application and use. It should be understood that in all of the accompanying drawings, similar reference numerals indicate the same or similar parts and features. The various drawings only schematically represent the concepts and principles of the embodiments of the present invention, and do not necessarily show the specific dimensions and proportions of the various embodiments of the present invention. Specific parts in specific drawings may be exaggerated to illustrate the relevant details or structures of the embodiments of the present invention.
[0048] Figure 1 FIG. 1 is a block diagram of an automatic detection system 1 according to an embodiment of the present invention. Figure 1 As shown, the automatic detection system 1 includes a motion platform 10, an interaction device 20, an environment perception device 30, a defect detection device 40, a processing device 50 and a memory 60. The interaction device 20, the environment perception device 30, the defect detection device 40, the processing device 50 and the memory 60 can all be mounted on the motion platform 10.
[0049] The motion platform 10 may be equipped with a driving motor and a plurality of running wheels, and may be able to move stably. The running wheels of the motion platform 10 may include a driving wheel that drives the motion platform 10 to move and a driven wheel that moves accordingly with the movement of the driving wheel.
[0050] The interactive device 20 provides an interactive interface between the automatic detection system 1 and the operator. After the automatic detection system 1 is turned on, the operator can start or stop the operation of the automatic detection system 1, select a detection mode, query detection records, etc. through the interactive device 20.
[0051] The environment perception device 30 is configured to perceive the environment in which the motion platform 10 is located during the movement of the motion platform 10, obtain the environmental data of the motion platform 10, and determine the position and posture of the motion platform 10 and the device carried thereon in real time, so as to be used for positioning and navigation. The environment perception device 30 includes various sensors or detectors for sensing the environment, such as but not limited to, a motion camera 31, an inertial measurement unit (IMU) 32, a wheel odometer 33, an impact sensor 34, a light intensity sensor 35, an ultrasonic sensor 36, a laser radar 37, and a time of flight (TOF) sensor 38. The environment perception device 30 may also include other sensors or detectors. The motion camera 31 may be mounted on the motion platform 10 so that the position and posture of the motion camera 31 relative to the motion platform 10 is adjustable. In one example, the motion camera 31 may be mounted on the motion platform 10 via a motion-controllable pan-tilt platform so that the shooting position and posture of the motion camera 31 may be changed by the movement of the pan-tilt platform. The motion camera 31 is configured to capture an environmental image as a positioning image during the movement of the motion platform 10. Preferably, the motion camera 31 adopts a high-speed binocular camera. The environment sensing device 30 further includes a lighting device 39, which can be provided on the motion camera 31. In the case of insufficient light, for example, when the light intensity sensor 35 detects insufficient light, the lighting device 39 can be automatically activated to illuminate the area in which it is located, so as to improve the quality of the positioning image taken by the motion camera 31, thereby improving the positioning accuracy.
[0052] The IMU 32 and the wheel odometer 33 are configured to measure the motion, distance, and orientation of the motion platform 10 during the movement of the motion platform 10. The measurement data of the IMU 32 and the wheel odometer 33 and the positioning image taken by the motion camera 31 are processed and used to determine the position and posture of the motion platform 10 and the devices carried thereon.
[0053] The impact sensor 34 is used to detect collisions that occur during the motion of the motion platform 10. The ultrasonic sensor 36, the laser radar 37, and the TOF sensor 38 form an obstacle detection sensor of the environment sensing device 30, which is used to detect obstacles that the motion platform 10 may encounter during the motion, so as to be used for obstacle avoidance processing during the navigation process of the motion platform 10. The laser radar 37 can be a 2D or 3D laser radar. It should be noted that the obstacle detection sensor of the environment sensing device 30 may include at least one of the ultrasonic sensor 36, the laser radar 37, and the TOF sensor 38, or may also include other sensors in addition to the ultrasonic sensor 36, the laser radar 37, and the TOF sensor 38.
[0054] The defect detection device 40 is configured to detect and measure defects of a detection object (e.g., an enclosed space within an aircraft, such as a cargo hold or a passenger cabin). The defect detection device 40 includes a detection shooting device 41 and a three-dimensional measuring device 42. The detection shooting device 41 is configured to capture detection images of the detection object at predetermined time intervals while the motion platform 10 moves within the area of the detection object (e.g., an enclosed space within an aircraft, such as a cargo hold), and store the captured detection images in a memory 60. The defect detection device 40 is mounted on the motion platform 10 so that the posture of the detection shooting device 41 relative to the mounting platform 10 is adjustable. The three-dimensional measuring device 42 is used to measure the three-dimensional size of the defect.
[0055] The memory 60 is used to store the identification number of the object to be detected and its corresponding digital machine model (DMU) data, learning data, detection rules and detection records.
[0056] The processing device 50 is configured to communicate with the motion platform 10, the interaction device 20, the environment perception device 30, the defect detection device 40, and the memory 60. The processing device 50 includes a data acquisition and preprocessing unit 51, a control unit 52, and a detection analysis unit 53. The data acquisition and preprocessing unit 51 includes a detection object data acquisition module 511, a positioning data preprocessing module 512, a collision and obstacle avoidance preprocessing module 513, and a detection data preprocessing module 514. The detection object acquisition module 511 obtains the user's input from the interaction device 20, and obtains the data of the detection object (for example, the DMU data, learning data, detection rules, and detection records of the detection object, etc.) from the memory 60 according to the user's input (for example, the identification number of the detection object).
[0057] The positioning data preprocessing module 512 can obtain the positioning image data taken by the motion camera 31 and the measurement data of the IMU 32 and the wheel odometer 33 from the environment perception device 30 and perform preprocessing, and calculate the real-time position and posture of the motion platform 10 and the device carried thereon relative to the detection object in combination with the data of the detection object obtained by the detection object data acquisition module 511. The positioning image preprocessing includes image processing such as feature point extraction on the positioning image obtained from the motion camera 31, and preferably, it also includes texture optimization processing on the positioning image before extracting feature points from the positioning image to improve the quality of the positioning image, thereby improving the robustness of the navigation positioning calculation. The texture optimization processing on the positioning image includes image processing on the positioning image obtained from the motion camera 31 to extract image texture information, and when it is judged that the texture in the positioning image is sparse, for example, when the surface of the part photographed by the motion camera 31 is smooth or the texture of the positioning image is sparse due to reflection, the positioning image is textured to improve the quality of the positioning image, thereby improving the robustness of the navigation positioning and the automatic detection performed based on the navigation positioning.
[0058] The collision and obstacle avoidance preprocessing module 513 can obtain the measurement data of the impact sensor 34, ultrasonic sensor 36, lidar 37 and TOF sensor 38 from the environmental perception device 30 and perform preprocessing for detecting collisions and obstacles to perform anti-collision and obstacle avoidance processing.
[0059] The detection data preprocessing module 514 obtains detection data (including detection images captured by the detection shooting device 41 and three-dimensional data measured by the three-dimensional measuring device 42) from the defect detection device 40 and preprocesses the detection data, associates the detection data with the corresponding real-time posture, forms detection position metadata, and stores it in the memory 60.
[0060] The control unit 52 performs control based on various data acquired by the data acquisition and preprocessing unit 51 and the preprocessing results of various data, including the control of the motion platform 10 and the defect detection device 40. The control unit 52 includes a path generation module 521, a collision detection module 522, an obstacle detection module 523 and a control module 524. The control module 524 performs corresponding control based on the processing results of the path generation module 521, the collision detection module 522 and the obstacle detection module 523. The path generation module 521 generates the moving path of the automatic detection system 1 in real time according to the real-time posture determined by the preprocessing of the positioning data by the positioning data preprocessing module 512 and the data of the detection object acquired by the detection object acquisition module 511, and the control module 524 controls the movement of the motion platform 10, the posture of the motion camera 31 and the posture of the detection shooting device 41 based on the generated moving path. The collision detection module 522 detects the collision based on the preprocessing of the data of the impact sensor 34 by the collision and obstacle avoidance preprocessing module 513. The obstacle detection module 523 detects obstacles based on the processing of the data of the obstacle detection sensors (ultrasonic sensor 36, laser radar 37 and TOF sensor 38) by the collision and obstacle avoidance preprocessing module 513. Once a collision is detected or an obstacle is detected ahead, the path generation module 521 regenerates the moving path, and the control module 524 controls the motion platform 10 based on the generated new moving path to quickly eliminate the current collision or bypass the obstacle to avoid a collision.
[0061] The detection and analysis unit 53 determines the detection result based on the real-time posture determined by the positioning data preprocessing module 512, the DMU data of the detection object obtained by the detection object acquisition module 511 from the memory 60 and the corresponding detection specification of the detection object, and preprocesses the detection image captured by the detection shooting device 41 according to the detection data preprocessing module 514, and stores the detection result in the memory 60. The detection result includes whether there is a defect (for example, a pit or a crack) and the location, size (length, width, depth) and three-dimensional contour of the defect. When a defect is detected, the detection and analysis unit 53 can start the three-dimensional measuring device 42 to measure the defective part to obtain the three-dimensional size information of the defect. It should be noted that in this example, the detection and analysis unit 53 is used as a part of the processing device 50, and the three-dimensional measuring device 42 is used as a part of the defect detection device 40. However, the present invention is not limited to this. In other examples according to the present invention, the detection and analysis unit 53 and the three-dimensional measuring device 42 can be an integrated device.
[0062] Figure 2 A perspective view showing an example of an automatic detection system 1 according to the present invention, Figure 3 Shows Figure 2 A stereogram of the automatic detection system 1 in FIG. 1 with the outer cover 70 removed, and Figure 4 1 shows a bottom view of the automatic detection system 1. The interaction device 20, the environment perception device 30, the defect detection device 40, the processing device 50 and the memory 60 (in Figure 2 and Figure 3 The outer cover 70 is not marked in the figure) and is mounted on the motion platform 1, accommodated in the outer cover 70, and partially exposed from the outer cover 70. The outer cover 70 can be installed with multiple anti-collision strips. In the example shown in the figure, the outer cover 70 is installed with two anti-collision strips 71 and 72 spaced apart from each other, so that even if a collision occurs with the detection object, the impact on the devices in the outer cover 70 can be reduced, and the outer cover 70 can also be prevented from causing damage to the detection object that collides with it. In addition, the anti-collision strips 71 and 72 of the outer cover 70 can be embedded with impact sensors 34.
[0063] In the example shown in the figure, a plurality of travel wheels are installed at the bottom of the motion platform 10, including two driving wheels 11, 12 and four driven wheels 13-16. Figure 4As shown. The driving wheels 11 and 12 are connected to the output shaft of the driving motor (not shown in the figure), so that the motion platform 10 is driven to move under the drive of the driving motor. The driving wheels 11 and 12 are connected to the motion platform 10 via a support rod (not shown in the figure) with a shock absorber (for example, a shock absorbing spring) to reduce vibration. A steering link (not shown in the figure) is installed between the driving wheels 11 and 12, which can make the driving wheels 11 and 12 rotate in the direction of travel. In addition, a differential device 17 is installed on the connecting shaft between the driving wheels 11 and 12, so that the driving wheels 11 and 12 can move at different speeds to help achieve steering.
[0064] The driven wheels 13-16 can change the direction of movement accordingly according to the direction of movement of the driving wheels 11, 12. The driven wheels 13-16 are installed to be spaced apart from each other around the center of the bottom of the motion platform 10. In the example shown in the figure, the driven wheels 13-16 are all universal wheels, and are arranged at four corners around the center of the bottom of the motion platform 10. The driven wheels 13-16 can be controlled to be independently lifted and lowered, so that a part of the driven wheels 13-16 can be involved in the movement of the motion platform 10 as needed, thereby providing better passability. Preferably, the lifting and lowering of two driven wheels in the driven wheels 13-16 that are located at diagonal positions to each other are controlled simultaneously, so that the motion platform 10 can be stably supported. By means of the driving wheels 11, 12 and the driven wheels 13-16 arranged in this way, greater flexibility and stability can be provided for the movement of the motion platform 10.
[0065] The motion platform 10 is also provided with a handle 18. When the outer cover 70 is installed, the handle 18 is exposed from a corresponding opening in the upper part of the outer cover. The operator can carry the automatic detection system 1 by grasping the handle 18.
[0066] In the example shown in the figure, the interactive device 20 is installed on the motion platform 10 and exposed from the corresponding opening of the outer cover 70 for the operator to operate. The interactive device 20 is in the form of an electronic display screen, for example, a touch-type electronic display screen. The interactive device 20 can also be in other suitable forms.
[0067] In the example shown in the figure, the environment sensing device 30 is installed on the motion platform 10, wherein the motion camera 31 is a high-speed binocular camera. When installed in place, the impact sensor 34 is embedded in the anti-collision strips 71 and 72 of the outer cover 70, and the wheel odometer 33 is installed on the driving wheel 11.
[0068] The automatic detection system 1 has two detection modes, ie, a learning detection mode and an automatic detection mode.
[0069] The following will take the cargo hold of an aircraft as an example of a detection object to introduce an automatic detection method using the automatic detection system 1 according to the present invention.
[0070] Figure 5 FIG. 2 shows a flow chart of the automatic detection method according to the present invention. Figure 5 As shown, in step S1, the automatic detection system 1 is moved to the vicinity of the cargo door of the aircraft, and the automatic detection system 1 is turned on. The operator can turn on the automatic detection system 1 by using the power switch (not shown) on the automatic detection system 1. Then, in step S2, the operator starts the automatic detection system 1 by operating on the interface of the interactive device 20 to identify and confirm the detection object. In this process, the motion camera 31 of the environmental perception device 30 is aimed at the QR code on the cargo door of the aircraft for shooting. The data acquisition and preprocessing unit 51 of the processing device 50 acquires the QR code image captured by the motion camera 31, identifies the information in the QR code image and compares it with the data stored in the memory 60, identifies the information of the cargo hold to be detected, such as the model and number of the cargo hold, and displays the detected cargo hold information on the interface of the interactive device 20 for confirmation by the operator.
[0071] After the operator confirms the information of the cargo hold to be inspected, in step S3, the operator selects the detection mode on the interface of the interactive device 20. The operator can choose the learning detection mode or the automatic detection mode. If the operator selects the learning detection mode, step S4 is entered to turn on the learning detection mode. In the learning detection mode, the operator operates the automatic detection system 1 to complete the detection of the cargo hold. Specifically, in step S41, the operator enters the cargo hold to be inspected together with the automatic detection system 1, and the operator controls the movement of the automatic detection system 1 (the driving speed, driving path, and dwell time at a certain point of the motion platform 10) by himself, and operates the defect detection device 40 to perform detection, thereby performing learning detection. In the process of performing detection in the learning detection mode, the operator can input the detection mark point data through the interactive device 20. Specifically, the operator can record a specific detection part as a detection mark point through the interactive device 20, and input the detection result. The specific detection part can be a part that is prone to defects (for example, pits or cracks) based on the operator's experience, or it can be a part where defects have been detected before. The processing device 50 processes the positioning data and other environmental measurement data obtained from the environmental sensing device 30, identifies the moving path of the motion platform 10 during this process, and identifies the detection mark points recorded by the operator and the data related to the detection mark points (for example, the dwell time or movement speed of the motion platform 10, the shooting posture of the detection shooting device 41, etc.), and stores them in the memory 60 for use as learning data for the automatic detection mode, wherein the identified moving path of the motion platform 10 can be used as the target detection path of the automatic detection mode. In step S42, the learning ends, and the operation data in this process is stored in the memory 60 for use as learning data for the automatic detection mode.
[0072] If the operator selects the automatic detection mode, the process proceeds to step S5 to prepare for the automatic detection. Specifically, in step S51, the processing device 50 queries the memory 60 to confirm whether the learning data of the cargo hold is available. If the learning data of the cargo hold is stored in the memory 60, the target detection path for this detection is obtained in step S52, and the automatic detection mode is entered in step S55 to perform automatic detection. If there is no learning data of the cargo hold in the memory 60, in step S53, information is displayed on the interactive device 20, indicating that there is temporarily no learning data of the cargo hold in the automatic detection system 1, and the operator is asked to confirm whether to continue the automatic detection. If the operator chooses not to perform the automatic detection, a mode selection interface pops up on the interactive device 20 for the operator to reselect the learning detection mode or exit directly. If in step S53, the operator confirms to continue the automatic detection, in step S54, the control unit 52 of the processing device 50 will automatically generate the target detection path based on the DMU data and detection rules of the cargo hold acquired by the data acquisition and preprocessing unit 51. Then, the process proceeds to step S55 to perform automatic detection.
[0073] After the detection in the learning detection mode or the automatic detection mode is completed, the automatic detection system 1 returns to the starting point. In step S6, the detection analysis unit 53 of the processing device 50 analyzes the detection position metadata stored in the memory 60 according to the DMU data of the cargo hold, judges the detection result according to the cargo hold detection specification, and generates a detection report. Among them, after the detection is performed in the learning detection mode, if the operator has input the detection result through the interactive device 20 during the detection process, the detection analysis unit 53 can generate a detection report according to the input of the operator through the interactive device 20 and in combination with the DMU data of the cargo hold. The content of the detection report includes whether there is a defect, the type of the detected defect, the location of the detected defect, and the three-dimensional size information of the detected defect. In step S7, the part where defects (for example, pits or cracks) are detected is marked as a new detection mark point, and the data related to the new detection mark point (the residence time or movement speed of the motion platform 10, the shooting posture of the detection and shooting device 41, etc.) is stored in the memory 60 to update the learning data of the cargo hold according to the detection results of the cargo hold for the next automatic inspection.
[0074] In step S8, it is confirmed whether to continue the detection. If the detection is continued, steps S2 to S7 are repeated. If the detection is not continued, in step S9, the detection operation is terminated.
[0075] It should be noted that step S6 and step S7 may also be executed after the detection operation is finished, for example, after the automatic detection system 1 returns to the detection center, thereby reducing the amount of real-time calculation of the processing device 50.
[0076] Figure 6 Shows Figure 5 FIG. 5 is a flow chart of the automatic detection of step S55 in FIG. After the automatic detection is started, in step S521, the detection object acquisition module 511 of the data acquisition and preprocessing unit 51 of the processing device 50 calls the DMU data and available learning data of the cargo hold from the memory 60. Using the DMU data, the entrance of the cargo hold, the outline of the cargo hold, and the facilities in the cargo hold (e.g., ground locks, vents, etc.) can be identified during the movement, so that corresponding avoidance can be taken during the movement, and the minimum safety distance can be determined during the movement according to the identified cargo hold outline.
[0077] In step S522, the data acquisition and preprocessing unit 51 acquires the positioning image taken by the motion camera 31 and performs preprocessing, and combines the DMU data of the cargo hold to determine the current position of the automatic detection system 1 relative to the cargo hold, thereby determining the detection starting point. In step S523, based on the determined detection starting point, according to the target detection path of this automatic detection, combined with the DMU data of the cargo hold, the automatic detection system 1 is navigated and positioned, and the movement speed of the motion platform 10, the position and posture of the motion platform 10 and the devices carried thereon are controlled, so that the automatic detection system 1 moves along the target detection path and performs detection. In the process of navigation and positioning, the positioning data preprocessing module 512 of the processing device 50 performs texture optimization processing on the positioning image taken by the motion camera 31 to improve the quality of the positioning image, thereby improving the robustness of navigation and positioning.
[0078] During the navigation movement along the target detection path, as shown in step C1, it is determined whether the detection mark point is reached. If the detection mark point is reached, in step C11, according to the learning data corresponding to the detection mark point stored in the memory 60, the motion platform 10 and the motion camera 31 mounted thereon, and the position and posture of the detection shooting device 41 are controlled, and the three-dimensional measurement device 42 is made to measure the detection mark point.
[0079] During the navigation movement, obstacles can also be automatically identified. As shown in step C2, the obstacle detection module 523 of the control unit 52 determines whether an obstacle is detected. After confirming that the obstacle is detected, in step C21, based on the measurement data of the ultrasonic sensor 36, the laser radar 37 and the TOF sensor 38, the obstacle detection module 523 of the control unit 52 determines the three-dimensional size, position, etc. of the obstacle, and compares it with the DMU data of the cargo hold to determine whether the obstacle is an inherent facility in the cargo hold (for example, a ground lock). If it is determined that the obstacle is not an inherent facility in the cargo hold, the obstacle is identified as an abnormality in step C22, and its information is stored in the memory 60.
[0080] Next, based on the information of the identified obstacle, it is determined in step C23 whether the obstacle can be directly crossed. When it is determined that the obstacle can be crossed, in step C24, the control unit 52 controls the motion platform 10 to keep the motion direction unchanged and continue to move along the predetermined target detection path to cross the obstacle. In this process, a pair of driven wheels located at the diagonal position among the driving wheels 13-16 can be raised to improve the passability of the motion platform 10 while ensuring the stability of the motion platform 10. For example, Figure 4 The driven wheels 13 and 15 in the vehicle body are raised, so that the motion platform 10 moves through the driving wheels 11 and 13 and another pair of driven wheels 14 and 16. When it is determined that the obstacle cannot be crossed, in step C25, the motion platform 10 is controlled to bypass the obstacle to avoid a collision. For example, when it is detected that the height of the obstacle exceeds 5 cm and the slope exceeds 10 degrees, it is considered that the obstacle cannot be directly crossed and therefore needs to be bypassed. During the bypass process, a suitable bypassing solution can be selected in combination with the position of the motion platform 10 and the obstacle relative to the cargo hold. For example, the driving wheels 11 and 12 of the motion platform 10 are as follows: Figure 4 Turn as shown by arrow K1 in Figure 4 The driven wheels 11 and 12 are retracted as shown by the arrow K3 in the figure, and the driven wheels 13-16 are turned as shown by the arrow K2 or the arrow K4. In this process, a pair of driven wheels located in the diagonal position among the driven wheels 13-16 can also be raised. For example, when the driving wheels 11 and 12 are Figure 4 When turning in the direction shown by the arrow K1 in FIG. 1 , the driven wheels 13 and 15 can be raised.
[0081] During the navigation movement, it is also possible to detect collisions and quickly mitigate the collisions that occur. In step C3, once the impact sensor 34 of the environment sensing device 30 detects a collision, the control unit 52 controls the motion platform 10 to brake the driving wheels 11 and 12 urgently and control them to detour, thereby quickly mitigating the collision that occurs. In addition, in the automatic detection system 1, anti-collision strips 71 and 72 (such as Figure 2 Even if a collision occurs, on the one hand, the adverse effect of the impact of the collision on the devices carried on the motion platform 10 can be reduced, and on the other hand, no damage will be caused to the cargo hold being inspected.
[0082] In step S524, it is determined whether the detection end point has been reached. If the detection end point has not been reached, the navigation movement is continued and the detection shooting is performed. If the detection end point has been reached, in step S525, the automatic detection ends.
[0083] During the navigation movement, the data acquisition and preprocessing unit 51 of the processing device 50 fuses the data from multiple sensors and uses a simultaneous localization and mapping (SLAM) positioning solution to determine the position and posture of the motion platform 10 and the devices carried thereon in real time, and the control unit 52 controls the motion platform 10 and the devices carried thereon accordingly.
[0084] Figure 7 FIG. 1 is a schematic block diagram of a SLAM-based positioning solution for multi-sensor data fusion according to an example of the present invention. In the example shown in the figure, the front-end processing of the SLAM-based positioning solution adopts a feature point method. Figure 7 As shown, first, the data of each sensor is preprocessed. The positioning image taken by the motion camera 31 (binocular camera) is automatically texture optimized, and feature points are extracted from the optimized positioning image data. The IMU data measured by the IMU 32 is preprocessed and the error is calculated. The data of the wheel odometer 33 is dead reckoning. The positioning image data of the motion camera 31 is fused with the IMU data, and the IMU data is fused with the data of the wheel odometer 33, and then positioning and composition are performed on this basis. Specifically, on the basis of the data fusion of multiple sensors, feature point statistics and matching are performed, and loop detection is performed, and then the real-time posture of the platform 10 is obtained after back-end optimization processing. In addition, in the positioning and composition processing, repositioning is also performed according to the obstacle information identified by the laser radar data, TOF sensor data and ultrasonic data to achieve the purpose of obstacle avoidance. In the above positioning scheme, by fusing the data of multiple sensors and performing loop detection, the cumulative error can be eliminated, the posture drift can be minimized, and accurate navigation positioning can be achieved. The position and posture of the device carried by the motion platform 10 can be calculated based on the relative position between the device and the motion platform 10. During the detection process, the calculated real-time position and posture of the detection and shooting device 41 is associated with the detection image captured by the detection and shooting device 41 to form detection position metadata, which is stored in the memory 60.
[0085] Figure 8 shows a schematic diagram of the detection of the automatic detection system 1 in the cargo hold V, Figure 8 The dotted line in shows the detection field of view of the automatic detection system 1 .
[0086] The automatic detection system 1 and the automatic detection method according to the present invention can flexibly select the detection mode as needed, and can learn based on the learning detection mode of manual operation, combine the operator's experience operation data into the automatic detection mode, and can effectively perform automatic detection, ensuring the accuracy of the detection result. Once entering the automatic detection mode, the automatic detection system 1 and the automatic detection method can automatically perform detection without the operator performing further remote operation control. The automatic detection system 1 and the automatic detection method according to the present invention can accurately perform real-time positioning and navigation based on the measurement data of multiple sensors in the detection process, combined with the DMU data of the detection object, thereby improving the accuracy of positioning and navigation, thereby further improving the accuracy of the detection result. In addition, the automatic detection system 1 and the automatic detection method can detect obstacles on the motion path, and can avoid obstacles, reduce or avoid collisions. In addition, anti-collision strips 71 and 72 are installed on the outer cover 70 of the automatic detection system 1, even in the event of a collision, the impact caused by the collision can be reduced, reducing the adverse effects on the various devices of the automatic detection system 1, and can also avoid the collision of the automatic detection system 1. Damage to the detection object.
[0087] The automatic detection system 1 and the automatic detection method according to the present invention are introduced above in conjunction with the accompanying drawings, and the application of the automatic detection system and the automatic detection method according to the present invention is introduced by taking the cargo hold of an aircraft as an example of the detection object. However, the above example should not be used as a limitation to the automatic detection system and the automatic detection method according to the present invention. The automatic detection system and the automatic detection method according to the present invention can also be applied to the detection of other application scenarios (for example, the cargo hold of a ship).
[0088] Here, exemplary embodiments of the present invention have been described in detail, but it should be understood that the present invention is not limited to the specific embodiments described and shown in detail above. Without departing from the spirit and scope of the present invention, those skilled in the art can make various modifications and variations to the present invention. All these modifications and variations fall within the scope of the present invention. Moreover, all components described herein can be replaced by other technically equivalent components.
Claims
1. An automatic detection system for a closed space, the automatic detection system comprising: An interaction device, the interaction device providing interaction between an operator and the automatic detection system; A motion platform capable of moving within the enclosed space; An environment sensing device, wherein the environment sensing device is configured to obtain environment data of the motion platform during the movement of the motion platform; A defect detection device, the defect detection device is configured to detect the enclosed space to generate detection data, the defect detection device includes a detection and shooting device; A memory storing digital prototype data of the closed space; as well as a processing device, the processing device being in communication with the motion platform, the interaction device, the environment sensing device, the defect detection device, and the memory, and being configured to process the environment data of the environment sensing device to control the motion platform and the defect detection device, and to process the detection data generated by the defect detection device to generate a detection report, It is characterized in that the interaction device, the environmental perception device, the defect detection device, the memory and the processing device are carried on the motion platform, and the interaction device can be operated to identify the enclosed space and enable the automatic detection system to automatically perform detection in an automatic detection mode according to the digital prototype data of the enclosed space.
2. The automatic detection system for a closed space according to claim 1, wherein: The processing device comprises: A data acquisition and preprocessing unit, the data acquisition and preprocessing unit is configured to acquire digital prototype data of the enclosed space from the memory, acquire environmental data of the environmental sensing device and detection data of the defect detection device, and preprocess the environmental data and the detection data; a control unit configured to communicate with the data acquisition and pre-processing unit to determine a target detection path; and a detection and analysis unit, the detection and analysis unit being configured to communicate with the data acquisition and preprocessing unit to analyze the detection results and generate a detection report, Wherein, the processing device is configured to: in the automatic detection mode, The data acquisition and preprocessing unit searches the memory for available learning data, wherein the learning data includes a target detection path and detection mark point data of the motion platform; When the data acquisition and preprocessing unit searches for available learning data, the control unit acquires a target detection path from the learning data; when the data acquisition and preprocessing unit does not search for available learning data, the control unit automatically generates a target detection path according to the digital prototype data of the enclosed space acquired by the data acquisition and preprocessing unit; and The control unit controls the motion platform to move along the target detection path.
3. The automatic detection system for a closed space according to claim 2, wherein: The interactive device can also be operated to cause the automatic inspection system to perform inspection in a learning inspection mode, in which the operator operates the motion platform and the defect inspection device and inputs inspection mark point data via the interactive device; as well as The processing device is configured such that, in the learning detection mode, the control unit communicates with the data acquisition and preprocessing unit to automatically identify the motion path and detection mark point data of the motion platform, and stores the identified motion path and detection mark point data of the motion platform in the memory for use as learning data for the automatic detection mode.
4. The automatic detection system for a closed space according to claim 2, wherein: The detection analysis section is configured to update learning data for the automatic detection mode of the closed space according to a detection result of the closed space.
5. The automatic detection system for a closed space according to claim 2, wherein: The environment sensing device comprises: a motion camera, the motion camera being configured to capture positioning images while moving with the motion platform; and an inertial measurement unit and a wheel odometer, the inertial measurement unit and the wheel odometer being configured to measure the motion of the motion platform, Wherein, the processing device is configured as follows: the data acquisition and preprocessing unit preprocesses the positioning image taken by the motion camera and the measurement data of the inertial measurement unit and the wheel odometer, so that the control unit can determine the position and posture of the motion platform.
6. The automatic detection system for a closed space according to claim 5, wherein: The data acquisition and preprocessing unit is configured to perform texture optimization processing on the positioning image.
7. The automatic detection system for a closed space according to claim 5, wherein: The environment sensing device further includes a light intensity sensor, and the automatic detection system further includes a lighting device, wherein the lighting device is disposed on the motion camera or the defect detection device; and The processing device is configured to selectively activate the lighting device according to the detection result of the light intensity sensor.
8. The automatic detection system for a closed space according to claim 5, wherein: The environment sensing device also includes: an impact sensor for detecting a collision of the automatic detection system; and an obstacle detection sensor configured to detect an obstacle on a motion path of the motion platform, the obstacle detection sensor comprising at least one of an ultrasonic sensor, a laser radar, and a time-of-flight sensor; and The processing device is configured such that: the data acquisition and preprocessing unit acquires the measurement data of the impact sensor and the obstacle detection sensor and performs preprocessing for the control unit to control the movement of the motion platform.
9. The automatic detection system for a closed space according to any one of claims 1 to 8, wherein: The automatic detection system further comprises an outer cover, which is mounted on the motion platform to at least partially accommodate a device carried by the motion platform inside the outer cover, and an anti-collision strip is mounted on the outer cover.
10. The automatic detection system for a closed space according to any one of claims 1 to 8, wherein: The motion platform is equipped with two driving wheels and a plurality of driven wheels, and the plurality of driven wheels are configured to be independently controllable.
11. The automatic detection system for a closed space according to any one of claims 1 to 8, wherein: The enclosed space is a cargo hold or a passenger cabin of an aircraft.
12. An automatic detection method for a closed space, the automatic detection method is performed by an automatic detection system, the automatic detection system comprises an interaction device, a motion platform, an environment sensing device, a defect detection device, a memory and a processing device, wherein: The interaction device, the environment perception device, the defect detection device, the memory, and the processing device are mounted on the motion platform, and the processing device communicates with the interaction device, the motion platform, the environment perception device, the defect detection device, and the memory. Characterized in that the automatic detection method comprises the following steps: operating the interactive device to identify and confirm the enclosed space; and operating the interactive device to enable the automatic detection system to automatically perform detection in an automatic detection mode according to the digital prototype data of the enclosed space; and Automatically generate test reports.
13. The automatic detection method for a closed space according to claim 12, wherein: When performing detection in the automatic detection mode, the automatic detection method includes: searching the memory for digital prototype data of the enclosed space; Searching the memory for available learning data, wherein the learning data includes a target detection path and detection mark point data of the motion platform; When available learning data is found, a target detection path is obtained from the learning data; when available learning data is not found, a target detection path is automatically generated according to the digital prototype data of the closed space; and The motion platform is controlled to move along the target detection path, and the defect detection device is controlled to detect the enclosed space to generate detection data.
14. The automatic detection method for a closed space according to claim 13, wherein: The automatic detection method further comprises: operating the interactive device so that the automatic detection system performs detection in a learning detection mode; and Wherein, when performing detection in the learning detection mode, the automatic detection method includes: The operator operates the automatic detection system to control the movement of the motion platform, controls the defect detection device to generate detection data, and inputs the detection mark point data through the interactive device; The processing device acquires the environmental data of the environmental perception device, and automatically identifies the motion path and detection mark point data of the motion platform, and stores the identified motion path and detection mark point data of the motion platform in the memory for use as learning data for the automatic detection mode.
15. The automatic detection method for a closed space according to claim 13, wherein: The environment perception device includes a motion camera, an inertial measurement unit and a wheel odometer; The automatic detection method comprises: processing the positioning image taken by the motion camera and the measurement data of the inertial measurement unit and the wheel odometer to calculate the real-time position and posture of the motion platform.
16. The automatic detection method for a closed space according to claim 15, wherein: The environment perception device further includes a super obstacle detection sensor and an impact sensor, wherein the obstacle detection sensor includes at least one of an acoustic sensor, a laser radar, and a time-of-flight sensor; as well as When the detection is performed in the automatic detection mode, the automatic detection method further includes: controlling the movement of the motion platform according to the measurement results of the obstacle detection sensor and the impact sensor.
17. The automatic detection method for a closed space according to claim 15, wherein: The environment sensing device further includes a light intensity sensor, and the automatic detection system further includes a lighting device, and the lighting device is arranged on the motion camera or the defect detection device; When the detection is performed in the automatic detection mode, the automatic detection method further includes: when the light intensity sensor detects insufficient light, automatically starting the lighting device.
18. The automatic detection method for a closed space according to claim 13, wherein: The defect detection device includes a detection and shooting device. When the detection is performed in the automatic detection mode, the automatic detection method includes: when reaching the detection mark point, controlling the speed of the motion platform and the posture of the detection and shooting device according to the learning data.
19. The automatic detection method for a closed space according to any one of claims 13 to 18, wherein: The automatic detection method further includes: updating learning data for the automatic detection mode of the enclosed space according to the detection result of the enclosed space.
20. The automatic detection method for a closed space according to any one of claims 12 to 18, wherein: The motion platform is equipped with two driving wheels and a plurality of driven wheels; When performing the inspection in the automatic inspection mode, the automatic inspection method further includes raising some of the plurality of driven wheels.
Citation Information
Patent Citations
Collaborative robot for visually inspecting an aircraft
CN105873825A
Automatic closed chamber detection device based on real-time image acquisition technology and method
CN105973308A
Livestock and poultry information sensing robot based on autonomous navigation and map construction method
CN109900280A
Indoor environment detection system and method based on wheeled robot
CN111813125A
Building defect detection method and device
CN113112098A
