Method and system for realizing software automatic adaptation based on hardware self-inspection and multifunctional robot
Through the hardware self-test method, the problem of low efficiency of hardware reduction and software adaptation in the existing technology is solved, and the flexibility and efficient adaptation of multi-function robot software is realized to meet the needs of users' arbitrary hardware selection. .
Patent Information
- Application Number
- CN202411392894.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2025-01-14
AI Technical Summary
Existing robot software needs to be adapted and developed according to the hardware module in advance, resulting in the problem of hardware reduction in cost reduction when users do not need all functions.
Through hardware self-test methods, such as the self-test of microphone arrays, face recognition cameras and starlight cameras, the software functions are automatically adapted to ensure that the hardware exists and retains the corresponding functions when there is no fault, and the function is hidden when there is no fault.
It realizes the flexibility and high efficiency of automatic software adaptation, allowing users to select configurations based on hardware, reduces the need for software updates, and meets the arbitrary increase and decrease requirements of hardware.
Smart Images
Figure CN119311494A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method, a system and a multifunctional robot for realizing automatic software adaptation based on hardware self-checking, and belongs to the technical field of intelligent robots. Background Art
[0002] At present, most robots on the market are single-function robots, such as patrol / inspection robots, food delivery robots, etc. In terms of the hardware configuration of the robot, taking a robot with a chassis as an example, the chassis will have power management functions such as obstacle avoidance, mapping and automatic recharging, and the sensors of the corresponding functions need to have radar, ultrasound, 3D cameras, etc. If you want to communicate with the robot, the robot host computer needs to be equipped with a hardware module with voice acquisition, such as a microphone array and other voice acquisition modules; to complete face recognition related functions, a visible light image acquisition camera module is required. If there is an automatic patrol / inspection function, a starlight camera or other corresponding types of cameras or other types of sensors such as a gimbal is required; and the software of the robots on the market is also adapted and developed based on the hardware modules that have been reserved in advance, but there is a situation. If multiple functions are developed on a robot, various hardware modules need to be adapted in advance, but in reality, some customers do not need so many functions, or want to remove some unused functions, so as to achieve a robot with a low price and satisfactory software functions by reducing the hardware. In this case, the robot software needs to plan in advance the ability to automatically adapt the corresponding software functions to different hardware combinations. Summary of the invention
[0003] The present invention aims to solve the various problems mentioned above, and further provide a method, system and multifunctional robot for realizing automatic software adaptation based on hardware self-checking.
[0004] To solve the above problems, the technical solution adopted by the present invention is: In the first aspect, the present invention discloses a method for realizing automatic software adaptation based on hardware self-test, which can complete the selection and adaptation of software through the self-test of hardware functions. The method is special in that it includes a microphone array self-test method, and the steps are as follows: Step 11: Play an audio file containing a certain number of wake-up words; Step 12: Count the number of wake-up words of the microphone array through the serial port; Step 13: Compare the number of wake-up words in the played audio file and the number of received wake-up words to see if they are equal. If they are equal, it means that the microphone array exists and is not faulty, and further proceed to step 14; if they are not equal, it means that the microphone array has a fault, and skip step 14 and go directly to step 15; Step 14: Adapt the software of the microphone array and retain its voice conversation related functions; Step 15: Hide voice conversation related functions.
[0005] Furthermore, the method for realizing automatic software adaptation based on hardware self-checking also includes a face recognition camera self-checking method, and the specific steps are as follows: Step 21: Open the ID number of the front camera through the camera interface of the Android system, check whether the interface return value is correct, and then determine whether the front camera, i.e., the face recognition camera, exists; If the interface return value is correct, it indicates that the face recognition camera exists, and further execute step 22; If the interface return value is incorrect, it means that the face recognition camera does not exist, which means the self-test has failed, and go to step 24; Step 22: Obtain the video stream of the face recognition camera. A person stands in front of the camera, and the video stream detects the face through the face detection algorithm. If the number of detected faces is not equal to 0 and the number is correct, it means that the face detection algorithm has detected a face, and the number of detected faces and the face positions are fed back via a screen display or voice, and step 23 is continued; If the number of detected faces is not equal to 0 but the number of detected faces is incorrect, it indicates that there is a fault in the face detection and recognition function, and step 24 is directly executed; Step 23: Adapt the relevant software for face detection / recognition and retain the relevant functions of face detection / recognition; Step 24: Hide face detection and recognition related functions.
[0006] Furthermore, the face detection algorithm used is the ArcSoft face detection algorithm or the Seetaface6 face detection algorithm.
[0007] Furthermore, the method for realizing automatic software adaptation based on hardware self-checking also includes a starlight camera self-checking method, comprising the following steps: Step 31: Open the ID number of the rear camera through the camera interface of the Android system and check whether the interface return value is correct to determine whether the starlight camera exists; If the interface return value is correct, it means that the starlight camera exists, and then proceed to step 32; If the return value is -1, it indicates that the rear camera fails to be opened, saying that the starlight camera is damaged or does not exist, and proceed to step 34; Step 32: Obtain the video stream of the starlight camera, place the ignited flame in front of the starlight camera, and detect the video stream of the starlight camera through the fireworks algorithm detection; If a fireworks alarm message is detected, it is said that there is no problem with the starlight camera and the corresponding fireworks algorithm, and the execution continues to step 33; If flames are seen on the screen, but the system does not have any fireworks alarm message prompts or voice prompts, it means that there are problems with the starlight camera and fireworks algorithm, and then execute step 34; Step 33: Adapt the relevant software of intelligent inspection / patrol and retain the relevant functions of intelligent patrol / patrol; Step 34: Hide the related functions of smart patrol / inspection.
[0008] Furthermore, the smoke and fire detection algorithm is trained via the YOLO5 target detection model.
[0009] In a second aspect, the present invention discloses a system for realizing automatic software adaptation based on hardware self-checking, wherein the system comprises a microphone array self-checking module.
[0010] Furthermore, the system also includes a face recognition camera self-check module and / or a starlight camera self-check module.
[0011] In a third aspect, the present invention discloses a multifunctional robot, which realizes automatic adaptation of software functions by calling the above-mentioned method for realizing automatic software adaptation based on hardware self-check in the power-on navigation function.
[0012] The method, system and multifunctional robot for realizing automatic software adaptation based on hardware self-check of the present invention are more flexible and have higher adaptation efficiency than the current software configuration of multifunctional robots, and can meet the user's arbitrary selection of robot hardware. Once the software is developed, the configuration of the hardware can meet any increase or decrease, and there is no need to modify and compile the software again for the increase or decrease of hardware, and update the robot software. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 : Flowchart of a method for realizing automatic software adaptation based on hardware self-checking in Example 1. DETAILED DESCRIPTION
[0014] The present invention will be described in detail below with reference to the accompanying drawings.
[0015] Example 1 A method for realizing automatic software adaptation based on hardware self-checking in this embodiment has the following process: Figure 1 As shown, it includes a microphone array self-test method, a face camera self-test method and a starlight camera self-test method; wherein: The microphone array self-test method comprises the following steps: Step 11: Play a pre-recorded audio file, such as an MP3 or WAV format audio file, and place the audio file in the system's assets directory. The audio file has a certain number of wake-up words preset; Step 12: When playing the audio file, the system counts the number of wake-up words of the microphone array through the serial port; Step 13: Compare the number of wake-up words in the played audio file and the number of received wake-up words to see if they are equal. If they are equal, it means that the microphone array exists and is not faulty, and further proceed to step 14; if they are not equal, it means that the microphone array has a fault, and skip step 14 and go directly to step 15; Step 14: Adapt the software of the microphone array and retain its voice conversation related functions; Step 15: Hide voice conversation related functions.
[0016] In this embodiment, the robot's host computer system is the Android 9.0 system. For the Android system, the IDs of its front camera and rear camera are different. The face recognition camera is the front camera of the system, and the starlight camera is a hardware device used to perform the patrol / inspection task video stream push function. It is located on the robot's chest and belongs to the rear camera in the system.
[0017] The face recognition camera self-test method comprises the following steps: Step 21: Open the ID number of the front camera through the camera interface of the Android system, check whether the interface return value is correct, and then determine whether the front camera, i.e., the face recognition camera, exists; If the interface return value is correct, it indicates that the face recognition camera exists, and further execute step 22; If the interface return value is incorrect, it means that the face recognition camera does not exist, which means the self-test has failed, and go to step 24; Step 22: Obtain a video stream of a face recognition camera. A person stands in front of the camera. The video stream detects the face through a face detection algorithm. In this embodiment, the face detection algorithm used is the ArcSoft face detection algorithm or the Seetaface6 face detection algorithm. If the number of detected faces is not equal to 0 and the number is correct, it means that the face detection algorithm has detected a face, and the number of detected faces and the position of the faces are fed back via screen display (displayed on the robot's screen) or voice (the number of detected faces is reported via voice), and step 23 is continued; If the number of detected faces is not equal to 0 but the number of detected faces is incorrect, it indicates that there is a fault in the face detection and recognition function, and step 24 is directly executed; Step 23: Adapt the relevant software for face detection / recognition and retain the relevant functions of face detection / recognition; Step 24: Hide face detection and recognition related functions.
[0018] The starlight camera self-test method comprises the following steps: Step 31: Open the ID number of the rear camera through the camera interface of the Android system and check whether the interface return value is correct to determine whether the starlight camera exists; If the interface return value is correct, it means that the starlight camera exists, and then proceed to step 32; If the return value is -1, it indicates that the rear camera fails to be opened, saying that the starlight camera is damaged or does not exist, and proceed to step 34; Step 32: Obtain the video stream of the starlight camera, light a match / lighter, place it in front of the starlight camera, and detect the video stream of the starlight camera through a fireworks algorithm, wherein the fireworks detection algorithm is trained by the YOLO5 target detection model; If a fireworks alarm message is detected, it is said that there is no problem with the starlight camera and the corresponding fireworks algorithm, and the execution continues to step 33; If the flame of a match / lighter is seen on the screen, but the system does not have any fireworks alarm message prompt or voice prompt, it means that there are problems with the starlight camera and the fireworks algorithm, and then execute step 34; Step 33: Adapt the relevant software of intelligent inspection / patrol and retain the relevant functions of intelligent patrol / patrol; Step 34: Hide the related functions of smart patrol / inspection.
[0019] The above is just an example of several functional modules of the current robot, which can be extended to more. The software adapts based on whether the hardware self-check exists / or the hardware is fault-free. After the development is completed, the software does not need to be re-adapted due to the increase / reduction of hardware modules. It is simple and fast, suitable for robots on the market to be matched according to user selection without changing the software again.
[0020] Embodiment 2 The present embodiment discloses a system for realizing automatic software adaptation based on hardware self-test, the system comprising a microphone array self-test module, a face recognition camera self-test module and a starlight camera self-test module, the microphone array self-test module, the face recognition camera self-test module and the starlight camera self-test module are respectively used to realize automatic adaptation of the microphone array, the front camera and the rear camera and their corresponding software.
[0021] Embodiment 3 This embodiment relates to a multifunctional robot. When the user first powers on the robot, there will be a startup navigation function, which guides the user on how to set up the robot. In the startup navigation, there will be a corresponding hardware self-test function, which will automatically detect whether the robot's important hardware peripherals exist or have faults, etc. If the self-tested hardware exists / self-tests without faults, the system will retain the software functions that match this hardware. When the user selects the hardware self-test function in the startup navigation interface, the system responds to the user's operation and executes the microphone array self-test method, the face recognition camera self-test method, and the starlight camera self-test method of the present invention respectively.
[0022] The multifunctional robot of the present invention can automatically adapt the software according to the increase or decrease of hardware. Compared with the current software configuration of the multifunctional robot, it is more flexible and has higher adaptation efficiency. It can meet the user's arbitrary choice of robot hardware. Once the software is developed, the configuration of the hardware can meet any increase or decrease. There is no need to modify the software again for the increase or decrease of hardware, compile the software, and update the robot software.
Claims
1. A method for realizing automatic software adaptation based on hardware self-checking, which completes the selection and adaptation of software based on the self-checking of hardware functions, characterized in that: Contains microphone array self-test method, the steps are as follows: Step 11: Play an audio file containing a certain number of wake-up words; Step 12: Count the number of wake-up words of the microphone array through the serial port; Step 13: Compare the number of wake-up words in the played audio file and the number of received wake-up words to see if they are equal. If they are equal, it means that the microphone array exists and is not faulty, and further proceed to step 14; if they are not equal, it means that the microphone array has a fault, and skip step 14 and go directly to step 15; Step 14: Adapt the software of the microphone array and retain its voice conversation related functions; Step 15: Hide voice conversation related functions.
2. A method for realizing automatic software adaptation based on hardware self-checking as claimed in claim 1, characterized in that: It also includes the face recognition camera self-test method, the specific steps are as follows: Step 21: Open the ID number of the front camera through the camera interface of the Android system, check whether the interface return value is correct, and then determine whether the front camera, i.e., the face recognition camera, exists; If the interface return value is correct, it indicates that the face recognition camera exists, and further execute step 22; If the interface return value is incorrect, it means that the face recognition camera does not exist, indicating that the self-test has failed, and go to step 24; Step 22: Obtain the video stream of the face recognition camera. A person stands in front of the camera, and the video stream detects the face through the face detection algorithm. If the number of detected faces is not equal to 0 and the number is correct, it means that the face detection algorithm has detected a face, and the number of detected faces and the face positions are fed back via a screen display or voice, and step 23 is continued; If the number of detected faces is not equal to 0 but the number of detected faces is incorrect, it indicates that there is a fault in the face detection and recognition function, and step 24 is directly executed; Step 23: Adapt the relevant software for face detection / recognition and retain the relevant functions of face detection / recognition; Step 24: Hide face detection and recognition related functions.
3. A method for realizing automatic software adaptation based on hardware self-checking as claimed in claim 2, characterized in that: The face detection algorithm used is the ArcSoft face detection algorithm or the Seetaface6 face detection algorithm.
4. A method for realizing automatic software adaptation based on hardware self-checking as claimed in claim 1, characterized in that: It also includes the Starlight Camera self-test method, which includes the following steps: Step 31: Open the ID number of the rear camera through the camera interface of the Android system and check whether the interface return value is correct to determine whether the starlight camera exists; If the interface return value is correct, it means that the starlight camera exists, and then proceed to step 32; If the return value is -1, it indicates that the rear camera fails to be opened, saying that the starlight camera is damaged or does not exist, and proceed to step 34; Step 32: Obtain the video stream of the starlight camera, place the ignited flame in front of the starlight camera, and detect the video stream of the starlight camera through the fireworks algorithm detection; If a fireworks alarm message is detected, it is said that there is no problem with the starlight camera and the corresponding fireworks algorithm, and the execution continues to step 33; If flames are seen on the screen, but the system does not have any fireworks alarm message prompts or voice prompts, it means that there are problems with the starlight camera and fireworks algorithm, and then execute step 34; Step 33: Adapt the relevant software of intelligent inspection / patrol and retain the relevant functions of intelligent patrol / patrol; Step 34: Hide the related functions of smart patrol / inspection.
5. A method for realizing automatic software adaptation based on hardware self-checking as claimed in claim 4, characterized in that: The smoke and fire detection algorithm is trained via the YOLO5 target detection model.
6. A method for realizing automatic software adaptation based on hardware self-checking as claimed in claim 2, characterized in that: It also includes the Starlight Camera self-test method, which includes the following steps: Step 31: Open the ID number of the rear camera through the camera interface of the Android system and check whether the interface return value is correct to determine whether the starlight camera exists; If the interface return value is correct, it means that the starlight camera exists, and then proceed to step 32; If the return value is -1, it indicates that the rear camera fails to be opened, saying that the starlight camera is damaged or does not exist, and proceed to step 34; Step 32: Obtain the video stream of the starlight camera, place the ignited flame in front of the starlight camera, and detect the video stream of the starlight camera through the fireworks algorithm detection; If a fireworks alarm message is detected, it is said that there is no problem with the starlight camera and the corresponding fireworks algorithm, and the execution continues to step 33; If flames are seen on the screen, but the system does not have any fireworks alarm message prompts or voice prompts, it means that there are problems with the starlight camera and fireworks algorithm, and then execute step 34; Step 33: Adapt the relevant software of intelligent inspection / patrol and retain the relevant functions of intelligent patrol / patrol; Step 34: Hide the related functions of smart patrol / inspection.
7. A system for realizing automatic software adaptation based on hardware self-checking, used to implement the method according to any one of claims 1 to 6, characterized in that: The system includes a microphone array self-test module.
8. A system for realizing automatic software adaptation based on hardware self-checking as claimed in claim 7, characterized in that: It also includes a face recognition camera self-test module and / or a starlight camera self-test module.
9. A multifunctional robot, characterized in that: In the startup navigation function, the multifunctional robot realizes automatic adaptation of software functions by calling the method for realizing automatic software adaptation based on hardware self-check as described in any one of claims 1 to 6.