A detection method and system for intelligent terminal voice interaction

Abstract

The present invention discloses a detection method and system for voice interaction of intelligent terminals, which combines a target detection algorithm and an autonomous navigation robot, and can test the intelligent functions of general intelligent devices. Under a tolerable misjudgment rate, it can replace a large amount of manpower and complete intelligent device testing through human-machine collaboration. The present invention uses the concept of human-machine collaboration to minimize manual operation and maximize intelligent operation. Robots and algorithm platforms replace manual work to complete boring and single tasks, and have standardized and retestable effects.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection, and in particular to an intelligent detection method and system for voice interaction of an intelligent terminal. Background Art

[0002] For intelligent devices such as intelligent washing machines and intelligent air conditioners, the existing testing methods still use manual methods, that is, manual voice or visual interaction with the intelligent devices being tested. However, the manual method has the following defects: First, due to human subjectivity, it will bring problems such as inconsistent voice broadcast content and inconsistent voice loudness (decibel), that is, lack of standardization; second, manual testing cannot record complete information during the test, and subsequent re-examination of the test results is impossible; third, facing the numerous types of intelligent devices and various models of intelligent devices, completing all the testing work will require a lot of manpower costs, and there are also problems of high labor intensity and low work efficiency.

[0003] For example, a Chinese invention patent (application number: CN211711342184.3) discloses an automatic testing method for a drum washing machine, which can collect the performance data of the drum washing machine in a simple way, making it easier for R&D personnel and experimental personnel to conduct on-site analysis, reducing the time for R&D and problem analysis, and is not limited by the site. However, in the specific implementation, there are still problems. Faced with a large number of home appliances and various home appliance models, completing all the testing work will require a lot of manpower costs, and there are also problems of high labor intensity and low work efficiency;

[0004] For example, the Chinese invention patent (CN112405561A) discloses a smart home appliance intelligence level test system based on voice interaction, including a robot body and a control system; wherein the robot body includes a magnetic navigation tracking mobile platform, an electric lifter and a servo steering gear; the control system includes a host computer, a slave computer, a voice interaction device, a route planning device and a data acquisition storage device; the test process is streamlined, the test method is strictly unified, highly consistent, and the test work is efficient. It adopts the idea of ​​modular integration, and adopts simple and economical existing resources on the premise of meeting the use requirements to save costs. However, in actual use, there are still certain shortcomings, such as the lack of good positioning and tracing capabilities for the intelligent ability test of smart home appliances, and the lack of interactive testing capabilities for voice and video of intelligent devices, and the robot lacks a good obstacle avoidance function when moving, which affects the adaptability of home appliance testing and is easily damaged by obstacles within the test range. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems of high labor intensity and low work efficiency in the prior art, and to provide a detection method and system for voice interaction of smart terminals.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] A detection method for intelligent terminal voice interaction includes the following steps:

[0008] S101. Testers formulate test tasks through the test system platform;

[0009] S102. After the intelligent robot autonomously navigates to the selected position according to the test task, it adjusts the corresponding operation and observation height through its own motion components;

[0010] S103. Control the intelligent device by playing voice commands through the intelligent robot's speakers, determine the intelligent control effect, and the camera records the corresponding video or photo to compare the control effect;

[0011] S104. Input the acquired video or photo into the corresponding text recognition algorithm or target detection algorithm;

[0012] S105. Match the recognized text target and voice command to obtain the test result and generate a test report.

[0013] Further, S101 includes: the tester selects the type of home appliance, the intelligent robot's arrival position, the audio playback height, the number of tests and the voice command to be tested on the system platform, and the system platform selects the commands that the intelligent device air conditioner does not have through the database, the database retrieves the commands through the SOL protocol, and at the same time the tester manually determines the overall test accuracy;

[0014] S102 includes: the intelligent robot simulates human voice playback, wherein the intelligent robot includes a host computer and a slave computer;

[0015] The host computer includes an industrial computer, an information collection mechanism, a laser radar, an audio microphone, and a movable wall mount; the movable wall mount is used to place the audio microphone, which is used to play voice commands and perform voice function testing on intelligent equipment;

[0016] The lower computer includes a single-chip microcomputer, an IMU, an ultrasonic sensor, a motor driver, and an encoder motor;

[0017] S103 includes: generating a manipulation effect text, and converting the manipulation effect text into a playable voice format through a cloud algorithm integrated in the single-chip microcomputer, wherein the manipulation effect text specifically includes three units: feature extraction, pattern matching, and reference pattern library, and the feature extraction extracts voice features by preprocessing the voice, extracts MFCC features by cutting off the first and last silences of the voiceprint of the sound wave, and extracts the MFCC features after framing the sound, the sound becomes an observation matrix, and uses initials and finals as phoneme sets to form the read words;

[0018] The system platform interacts with Dahua network cameras and intelligent robots to complete the video recording and storage corresponding to each voice command;

[0019] S104 includes: passing the video or picture corresponding to each voice instruction for text recognition into a text detection algorithm to obtain the coordinates of the text area in the picture, and then passing the coordinate information of the text area into the text recognition algorithm, the text recognition algorithm includes detecting and extracting the coordinate information, segmenting the picture of a single line of text by projecting histogram, segmenting the single line of text, and correcting the segmented characters by convolutional neural network to obtain corrected text;

[0020] The video or photo used for target detection is passed into the target detection algorithm. If the target is detected, it will be marked as a regular rectangle in the image and the text category information will be saved;

[0021] S105 includes: extracting keywords from the voice command through natural language processing, and matching the text target obtained by text recognition and the text category information obtained by target detection with the voice command keywords respectively, and marking as successful if the match is successful, otherwise as failed;

[0022] After all test tasks are completed, a test task report is generated.

[0023] Furthermore, S101 also includes: the tester inputs the voice command to be tested, the location that the intelligent robot is about to reach, the height of the sound playing, the type of the dialect played, and the number of tests in the system platform;

[0024] Among them, the input voice command will be converted into audio that can be played by the robot through an algorithm;

[0025] S102 includes: the intelligent robot can realize autonomous navigation to reach the position and height specified by the tester and play the command;

[0026] S103 includes: recording or photographing the intelligent device panel through an image acquisition device to complete the visual acquisition work of the intelligent device panel responding to the voice command;

[0027] S104 includes: obtaining the coordinates and categories of the corresponding targets by inputting the video stream or photos collected by the image acquisition device into the target detector yolov5, skipping frames to detect the video, and when the target is detected, the corresponding target will be marked by a rule box, and the target type will be stored in label[]. The same applies to the photos;

[0028] S105 includes: matching the detected text target with the voice command by keywords. If the match is successful, it means that the intelligent device has completed the feedback of the voice command. If the match fails, it means that the intelligent device has not completed the voice command. After the match fails, the labeling mechanism (3) will affix an unqualified label to the surface of the intelligent device that has not completed the voice command.

[0029] After all test tasks are completed, a test task report is generated.

[0030] The present invention also provides a detection system for a voice interaction detection method of an intelligent terminal, comprising a carrying plate, wherein an obstacle avoidance mechanism for obstacle avoidance and shock reduction is arranged on the lower surface of the carrying plate, walking mechanisms are installed at both ends of the obstacle avoidance mechanism, an adjustment mechanism is arranged on the upper surface of the carrying plate, an anti-fall mechanism for anti-fall protection is installed in the adjustment mechanism, the anti-fall mechanism is connected to a limiting mechanism, a lifting mechanism is installed on the top of the adjustment mechanism, an information collection mechanism is installed on the top of the lifting mechanism, and a broadcasting mechanism is arranged on one side of the information collection mechanism.

[0031] Furthermore, the obstacle avoidance mechanism includes a connecting base fixed to the supporting plate, and a group of mounting rods are symmetrically arranged at the front and rear ends of the connecting base, and each group of mounting rods is provided with 2, and a connecting block is installed at the other end of the mounting rod connected to the connecting base, and the cross-section of the mounting rod is an "H"-shaped structure, and the inner side of the mounting rod is hinged to one end of the first connecting rod, and the other end of the first connecting rod is provided with a second connecting rod, the first connecting rod and the second connecting rod are connected by a strong spring, and the second connecting rod is connected to the connecting base; the walking mechanism includes a driving motor fixed to the connecting block, the output end of the driving motor is connected to an output shaft, and the end of the output shaft is provided with a walking wheel, and the adjustment mechanism includes an installation box fixed to the supporting plate, a servo motor is installed on one side above the installation box, and the output end of the servo motor is connected to a transmission rod extending into the interior of the installation box, and fixed blocks are provided at both left and right ends of the transmission rod, and the fixed blocks are symmetrically arranged about the central axis of the stop block, and the outer surfaces of the fixed block and the stop block are both in a meshing structure.

[0032] Furthermore, a support platform is provided inside the installation box, and the anti-fall mechanism includes a movable rod slidably connected to the support platform, and two grooves are provided inside the movable rod, and a first movable block and a second movable block are respectively provided in the two grooves, and the movable rod and the first movable block, and the movable rod and the second movable block are connected by a first spring, and the first movable block and the second movable block are arranged opposite to each other, and an extrusion block is fixed on one side of the top of the movable rod, and the first movable block and the second movable block both form a sliding structure inside the movable rod, and the first movable block and the second movable block are both meshed and connected with the stop block. The limiting mechanism includes a vertical rod connected to the stop block, and a fixed rod passes through the inside of the vertical rod, and the vertical rod can rotate around the fixed rod, and a support plate is provided on the other end of the fixed rod connected to the vertical rod, and a spring connecting plate is also installed on the support plate, and a second spring is fixed on the spring connecting plate, and a block engaged with the stop block is provided on one side of the upper part of the vertical rod, and the other side is in contact with the second spring; the rotation angle of the lower part of the vertical rod is limited by the extrusion block.

[0033] Furthermore, the lifting mechanism includes a control rod meshingly connected to a fixed block, a mounting seat is fixed to the top of the control rod, a servo motor is installed at one end of the mounting seat, an output end of the servo motor is connected to a connecting rod with a threaded outer surface, the connecting rod passes through the interior of the mounting sleeve, and the control rod forms a lifting structure through the fixed block. The information collection mechanism includes a gear rotatably connected to the mounting sleeve, a rack is provided in front of the gear, a collection camera is installed on the top of the gear, and the gear forms a sliding structure on one side of the rack through the connecting rod. The broadcasting mechanism includes a wall-mounted body fixedly connected to the mounting seat, and a speaker is provided on one side of the wall-mounted body.

[0034] The present invention also provides another detection system for a voice interaction detection method of an intelligent terminal, comprising a walking mechanism, characterized in that a lifting mechanism for adjusting the height is arranged on the top of the walking mechanism, a labeling mechanism for marking is arranged on the top of the lifting mechanism, an image acquisition device for recording video clips is arranged on the top of the labeling mechanism, and audio devices for playing voice commands are arranged on both sides of the labeling mechanism.

[0035] Furthermore, the walking mechanism includes a fixed plate, the four corners of the bottom end of the fixed plate are fixedly connected with universal wheels, the fixed plate is embedded with laser radars around the four sides, the lifting mechanism includes a lifting frame fixedly connected to the fixed plate, the top of the lifting frame is provided with a lifting slot, the lifting slot is slidably connected with a lifting column, the top of the lifting column is fixedly connected with a control box, the front of the control box is provided with a placement port, one side of the lifting column is fixedly provided with a rack, the lifting frame is provided with a gear shaft rotatably connected to one side of the rack, the lifting frame is fixedly provided with a first stepper motor located at the bottom of the gear shaft, and the first stepper motor is meshed with the gear shaft through a gear;

[0036] Furthermore, the labeling mechanism includes a labeling box slidably connected to the placement port, the labeling box is internally rotatably connected to two winding disks, the labeling box is rotatably connected to a guide shaft located on one side of the winding disk, the labeling box is rotatably connected to a labeling column located on one side of the guide shaft, the labeling column is provided with a placement groove at the top and bottom, the placement groove is rotatably connected to a rotating roller, a slide is fixedly provided in the control box, a slide groove is provided at the top of the slide, a first electric telescopic rod located on one side of the slide groove is fixedly installed on the slide, a motor plate is slidably connected to the inside of the slide groove, one side of the motor plate is fixedly connected to the telescopic end of the first electric telescopic rod, a third stepping motor is provided at the top of the motor plate, a second stepping motor located on one side of the third stepping motor is provided on the motor plate, and bevel gears are fixedly provided at the top of the rotating shafts of the third stepping motor and the second stepping motor.

[0037] Furthermore, the image acquisition device includes a fourth stepper motor fixedly connected to the control box, the top end of the rotating shaft of the fourth stepper motor is fixedly connected to a connecting plate, the top end of the connecting plate is fixedly connected to a connecting seat, the top of the connecting seat is fixedly connected to a camera, a second electric telescopic rod located on one side of the connecting seat is fixedly provided on the connecting plate, the second electric telescopic rod is rotatably connected to the camera, the audio device includes a third electric telescopic rod rotatably connected to the control box, the telescopic end of the third electric telescopic rod is rotatably connected to a hanging wall, a speaker is fixedly connected to the hanging wall, the connection between the third electric telescopic rod and the hanging wall is rotatably connected to a fifth stepper motor, the fifth stepper motor is used to drive the hanging wall to rotate, the side of the third electric telescopic rod is rotatably connected to a sixth stepper motor, and the sixth stepper motor is used to drive the third electric telescopic rod to rotate.

[0038] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0039] 1. The obstacle avoidance mechanism is set up, which enables the device to have an obstacle avoidance function within a certain height range, replacing the direct control of walking operation in the prior art, and solves the damage to the precision instruments inside the device caused by high frequency or intense vibration during the walking of the device. At the same time, the obstacle avoidance structure has a self-resetting function, which makes the walking process of the device more stable and improves the practicability of the equipment.

[0040] 2. The adjustment mechanism can be used to conveniently control the lifting and sliding of the mounting base, so that the device can quickly reach the specified position and height, which is convenient for the control of the device and easy to use.

[0041] 3. During the lifting and lowering process of the mounting base, the lifting mechanism can be limited by the anti-fall mechanism to achieve the purpose of anti-fall protection and avoid damage caused by accidental falling of the equipment.

[0042] 4. The information collection mechanism can be set up to drive the collection camera to rotate while controlling the collection camera to slide left and right, so as to accurately capture the selected object and perform multi-directional information data scanning at the same time, making the result more accurate and enhancing the practicality of the device.

[0043] 5. Through the setting of labeling mechanism, the marking of unqualified equipment is realized, which is convenient for the staff to find unqualified products after the unified inspection is completed, reducing the staff's need to spend a lot of time to find unqualified products, and improving the practicality of the product. By slidingly connecting the labeling mechanism with the placement port, it is convenient to replace the label paper when there is no label paper, further improving the practicality of the device.

[0044] 6. The installed laser radar can prevent the device from colliding with unknown objects while moving, causing damage to the equipment, thereby improving the practicality of the product.

[0045] 7. By setting up the lifting device, the height of the control box can be adjusted, thereby realizing the detection of intelligent equipment at different heights, increasing the scope of equipment use and improving the practicability of the device.

[0046] 8. The second electric telescopic rod is provided, and the second electric telescopic rod has the function of adjusting the camera angle. The third stepping motor is provided to adjust the direction of the camera, which is convenient for adjusting the camera at will to achieve the best shooting angle.

[0047] 9. The present invention combines target detection algorithms and autonomous navigation robots to test the intelligent functions of general intelligent devices. Under a tolerable misjudgment rate, it can replace a large amount of manpower and complete intelligent device testing through human-machine collaboration.

[0048] 10. The present invention utilizes the concept of human-machine collaboration to minimize manual operations and maximize intelligent operations. Robots and algorithm platforms replace manual labor to complete boring and monotonous tasks, and have standardized and repeatable effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of one implementation of the intelligent detection method for voice interaction of an intelligent terminal provided in this application;

[0050] Figure 2 A flowchart of another implementation of the intelligent detection method for intelligent terminal voice interaction provided by the present application;

[0051] Figure 3 For implementation Figure 1 A schematic diagram of the overall front view structure of a detection system for voice interaction of an intelligent terminal according to the method;

[0052] Figure 4 For implementation Figure 1 An overall bottom-up schematic diagram of a detection system for voice interaction of a smart terminal according to the method;

[0053] Figure 5 For implementation Figure 1 A schematic diagram of the structure of an obstacle avoidance mechanism of a detection system for voice interaction of an intelligent terminal according to the method;

[0054] Figure 6 For implementation Figure 1 A schematic diagram of the gear and rack connection structure of the detection system for intelligent terminal voice interaction according to the method;

[0055] Figure 7 For implementation Figure 1 A schematic diagram of the structure of the lifting mechanism of the detection system for intelligent terminal voice interaction according to the method;

[0056] Figure 8 For implementation Figure 1 A schematic diagram of the structure of the adjustment mechanism of the detection system for intelligent terminal voice interaction according to the method;

[0057] Fig. 9 For implementation Figure 1 A schematic diagram of the structure of a limit mechanism of a detection system for voice interaction of an intelligent terminal according to the method;

[0058] Fig.10 For implementation Figure 1 A schematic diagram of the structure of the anti-fall mechanism of the detection system for intelligent terminal voice interaction according to the method;

[0059] Fig.11 For implementation Figure 1 A schematic diagram of the enlarged structure at point A in the detection system for voice interaction of an intelligent terminal according to the method;

[0060] Fig.12 For implementation Figure 2 The overall structural diagram of the detection system for intelligent terminal voice interaction of the method;

[0061] Fig.13 For implementation Figure 2 A plan view of a labeling mechanism of a detection system for voice interaction of a smart terminal according to the method;

[0062] Fig.14 For implementation Figure 2 A plan view of a labeling mechanism of a detection system for voice interaction of a smart terminal according to the method;

[0063] Fig.15 For implementation Figure 2 A schematic diagram of a wall-mounted connection structure of a detection system for voice interaction of an intelligent terminal according to the method;

[0064] Fig.16 For implementation Figure 2 A cross-sectional view of an image acquisition device of a detection system for voice interaction of an intelligent terminal according to the method;

[0065] Fig.17 For implementation Figure 2 A schematic diagram of the gear shaft connection of the detection system for intelligent terminal voice interaction according to the method;

[0066] Fig.18 For implementation Figure 2 A schematic diagram of the inner side structure of a control box of a detection system for voice interaction of an intelligent terminal according to the method;

[0067] Fig.19 For implementation Figure 2 Schematic diagram of the partial structure of the front side of the control box of the detection system for voice interaction of smart terminals according to the method. DETAILED DESCRIPTION

[0068] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0069] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0071] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0072] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the invention product is usually placed when in use, or the orientation or positional relationship commonly understood by those skilled in the art. Such terms are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0073] Embodiment 1:

[0074] like Figure 1 As shown, an intelligent detection method for voice interaction of an intelligent terminal comprises the following specific steps:

[0075] S101. Develop a test system platform. Testers develop test tasks through the system platform and judge the intelligence strength and working status of intelligent equipment through test tasks. It should be noted that the test system platform is specifically a test communication product with transmitting and receiving performance, which can be specifically set as an OTA test system platform.

[0076] S102. After the intelligent robot autonomously navigates to the selected position according to the multi-index test task, it adjusts the corresponding operation and observation height through its own motion components;

[0077] S103. Control the intelligent device by playing voice commands through the intelligent robot's speakers, determine the intelligent control effect, and the camera records the corresponding video or photo to compare the control effect;

[0078] S104. Input the acquired video or photo into the corresponding text recognition algorithm or target detection algorithm;

[0079] S105. Match the recognized text target and voice command to obtain the test result and generate a test report;

[0080] in:

[0081] S101 includes: the tester selects the type of home appliance (taking smart TV as an example in this embodiment), the arrival position of the smart robot, the height of the audio playback, the number of tests and the voice command to be tested on the system platform, and the tester determines whether it is the first test. If so, a task template is created based on the test system platform, and task parameters are collected through the test system platform. The instructions that the smart TV does not have are filtered through the database. The database retrieves the instructions through the SOL protocol, and the tester manually marks them at the same time to determine the overall test accuracy, so as to achieve the formulation of multi-indicator tasks;

[0082] If the tester determines that this is not the first test, a multi-indicator task will be directly formulated based on the test system platform;

[0083] S102 includes: the intelligent robot simulates human voice playback, wherein the intelligent robot includes a host computer and a slave computer, the host computer includes an industrial computer, an information collection mechanism 8, a laser radar, an audio microphone, and a movable wall, wherein the movable wall is used to place the audio microphone, the audio microphone is used to play voice commands, and is used to test the voice function of intelligent equipment, and the slave computer includes a single-chip microcomputer, an IMU, an ultrasonic sensor, a motor drive, and an encoder motor.

[0084] S103 includes: generating a control effect text, and converting the control effect text into a playable voice format through a cloud algorithm. The control effect text specifically includes three units: feature extraction, pattern matching, and reference pattern library. The feature extraction extracts voice features by preprocessing the voice, and extracts MFCC features by removing the beginning and end silences of the voiceprint of the sound wave and framing the sound. The sound becomes an observation matrix, and uses initial consonants and finals as phoneme sets to form the read words. The system platform interacts with Dahua network cameras and intelligent robots to complete the video recording and storage corresponding to each voice command.

[0085] S104 includes:

[0086] The video or picture corresponding to each voice command for text recognition is passed into the text detection algorithm to obtain the coordinates of the text area in the picture, and then the coordinate information is passed into the text recognition algorithm. The text recognition algorithm includes detecting and extracting the coordinate information, segmenting the picture of a single line of text through the projection histogram, segmenting the characters of the single line of text, and obtaining the corrected text after image processing through the convolutional neural network, and extracting the feature text information. The convolutional neural network can assist in the judgment of the extracted features by connecting each neuron with the neurons in the adjacent layer, and improve the accuracy of the algorithm feature extraction through the weighted judgment of the hidden layer neurons;

[0087] The video or photo used for target detection is passed into the target detection algorithm. If the target is detected, it will be marked as a regular rectangle in the image and the text category information will be saved;

[0088] And the coordinates of the extracted text area are shown in formula 1,

[0089] Where P(x, y) is the joint probability distribution function, and p(x) and p(y) are the marginal distribution functions of X and Y respectively;

[0090] S105 includes: extracting keywords from voice commands through natural language processing, and matching text targets obtained through text recognition and text category information obtained through target detection with the voice command keywords respectively; if the match is successful, it is marked as successful, otherwise it is marked as failed; after all test tasks are completed, a test task report is generated.

[0091] Embodiment 2:

[0092] like Figure 2 As shown, an intelligent detection method for voice interaction of an intelligent terminal includes:

[0093] S101. Develop a test system platform. Testers develop test tasks through the system platform and judge the intelligence strength and working status of intelligent devices through test tasks. Testers enter the voice command to be tested, the location of the intelligent robot, the height of the sound playback, the type of dialect played, and the number of tests in the platform.

[0094] The input voice commands will be converted into audio that can be played by the robot through an algorithm.

[0095] S102. After the intelligent robot autonomously navigates to the selected position according to the test task, it adjusts the corresponding operation and observation height through its own motion components; the intelligent robot simulates human voice playback, wherein the intelligent robot includes a host computer and a slave computer;

[0096] The host computer includes an industrial computer, an image acquisition device 41, a laser radar, an audio microphone, and a movable wall. The movable wall is used to place the audio microphone, and the audio microphone is used to play voice commands;

[0097] The lower computer includes a single-chip microcomputer, an IMU, an ultrasonic sensor, a motor driver, and an encoder motor;

[0098] The intelligent robot can autonomously navigate to the position and height specified by the tester and play instructions.

[0099] S103. Control the intelligent device by playing voice commands through the intelligent robot's speakers, judge the intelligent control effect, and record the corresponding video or photos with the camera to compare the control effect; record or shoot the intelligent device panel through the image acquisition device 41 to complete the visual acquisition work of the intelligent device panel's feedback on the voice command.

[0100] S104. Input the acquired video or photo into the corresponding text recognition algorithm or target detection algorithm; by inputting the video stream or photo captured by the image acquisition device 41 into the target detector yolov5, the coordinates and category of the corresponding target can be obtained, and the frame detection video is skipped. When the target is detected, the target will be marked by a rule box, and the target type will be stored in label[]. The same as above for the photo.

[0101] S105. Match the recognized text target and the voice command to obtain the test result and generate a test report; match the detected target category and the command by keywords. If the match is successful, it means that the intelligent device has completed the feedback of the voice command. If the match fails, it means that the intelligent device has not completed the voice command.

[0102] If the matching fails, the labeling agency will attach an unqualified label to the surface of the intelligent device that has not completed the voice command.

[0103] After all test tasks are completed, a test task report is generated.

[0104] Embodiment 3:

[0105] The solution in Example 1 is further introduced below in combination with a specific working method, as described below:

[0106] like Figure 3-Figure 11As shown in , this embodiment proposes an intelligent detection system for voice interaction of intelligent terminals, including a carrying plate 1. The carrying plate 1 is provided to realize the carrying and transportation of the device body. An obstacle avoidance mechanism 2 for obstacle avoidance and shock reduction is provided at the bottom of the carrying plate 1. The obstacle avoidance mechanism 2 is provided to enable the device to have an obstacle avoidance function within a certain height range, which can protect the precision instruments in the device. Both ends of the obstacle avoidance mechanism 2 are equipped with walking mechanisms 3 for moving the device. The walking mechanism 3 is provided to facilitate the control device to move to a selected position. An adjustment mechanism 4 is provided on the top of the carrying plate 1. The adjustment mechanism 4 is provided to facilitate the control device to reach a selected height. The inner side of the adjustment mechanism 4 is installed There is an anti-fall mechanism 5 for anti-fall protection. The anti-fall mechanism 5 can avoid the accidental falling of the device mechanism. A limiting mechanism 6 is arranged on the upper rear of the anti-fall mechanism 5. The limiting mechanism 6 can effectively improve the stability of the device operation. A lifting mechanism 7 for controlling the information collection height is installed on the top of the adjustment mechanism 4. The lifting mechanism 7 can adjust the information collection height to enhance the practicality of the device. The upper end of the lifting mechanism 7 is connected to an information collection mechanism 8. The information collection mechanism 8 can be used to collect data from all directions of the target product. A broadcasting mechanism 9 is arranged on the left side of the information collection mechanism 8. The broadcasting mechanism 9 can play voice commands in real time to facilitate device control.

[0107] As a preferred embodiment, on the basis of the above-mentioned manner, further, the obstacle avoidance mechanism 2 includes a connecting base 201 fixed to the carrying plate 1, two groups of mounting rods 202 are symmetrically arranged at the front and rear ends of the connecting base 201, and each group of mounting rods 202 is provided with two, and the other end of the mounting rod 202 connected to the connecting base 201 is provided with a connecting block 203, and the emergency obstacle avoidance of the device can be achieved by the mounting rod 202 which is hinged to the connecting base 201 and the connecting block 203, and the cross section of the mounting rod 202 is an "H"-shaped structure, and a first connecting rod 204 is hinged on the inner side of the mounting rod 202, and a second connecting rod 205 is arranged on the other end where the first connecting rod 204 is hinged to the mounting rod 202, and the first connecting rod 204 and the second connecting rod 205 are connected by a strong spring 206, and the second connecting rod 205 is connected to the connecting base 201, and the first connecting rod 204 is connected to the second connecting rod 205 through the second connecting rod 204. The internal expansion and contraction can be absorbed and offset by the strong spring 206 to ensure the stability of the device during operation. The walking mechanism 3 includes a driving motor 301 fixed to the connecting block 203, and the output end of the driving motor 301 is connected to the output shaft 302. The other end of the output shaft 302 fixed to the driving motor 301 is provided with a walking wheel 303, which is convenient for controlling the device to move to the selected position. The adjustment mechanism 4 includes an installation box 401 fixed to the supporting plate 1, and a servo motor 402 is installed on the upper right side of the installation box 401. The output end of the servo motor 402 is connected to a transmission rod 403, and fixed blocks 404 are provided on both ends of the transmission rod 403. The fixed blocks 404 are symmetrically arranged about the central axis of the stop block 405. The outer surfaces of the fixed blocks 404 and the stop block 405 are both in a meshing structure, which is convenient for controlling the operation of the lifting mechanism 7 and for controlling the device to reach the selected height.

[0108] As a preferred embodiment, on the basis of the above-mentioned method, further, a support platform is provided inside the installation box 401, and the anti-fall mechanism 5 includes a movable rod 501 slidably connected to the installation box 401, and two grooves are provided inside the movable rod 501, and a first movable block 502 and a second movable block 504 are respectively provided in the two grooves, and the movable rod 501 and the first movable block 502, and the movable rod 501 and the second movable block 504 are connected by a first spring 503, and the first movable block 502 and the second movable block 504 are arranged opposite to each other, and an extrusion block 505 is fixed to one side of the top of the movable rod 501, and the first movable block 502 and the second movable block 504 both constitute a sliding structure inside the movable rod 501, and the first movable block 502 and the second movable block 504 constitute a sliding structure inside the movable rod 501, and the first movable block 502 and the second movable block 504 are respectively provided. The movable blocks 504 are all meshed and connected with the stop block 405, which is convenient for driving the vertical rod 601 to deflect during the process of controlling the lifting mechanism 7 to move downward, so that the lifting mechanism 7 can move downward stably. The limiting mechanism 6 includes a vertical rod 601 that is engaged with the stop block 405, and a fixed rod 602 is penetrated inside the vertical rod 601. The vertical rod 601 can rotate around the fixed rod 602. The other end of the fixed rod 602 connected to the vertical rod 601 is provided with a support plate 603, and a spring connecting plate is also installed on the support plate 603. The second spring 604 is fixed on the spring connecting plate. A block that is engaged with the stop block 405 is provided on one side of the upper part of the vertical rod 601, and the other side is in contact with the second spring 604; the lower part of the vertical rod 601 is limited in rotation angle by the extrusion block 505. In the process of lifting the lifting mechanism 7, a safety limit can be provided for the lifting mechanism 7, which plays a protective role for the device.

[0109] As a preferred embodiment, on the basis of the above-mentioned method, further, the lifting mechanism 7 includes a control rod 701 meshed with the fixed block 404, a mounting seat 702 is fixed to the top of the control rod 701, a servo motor 703 is installed at the left end of the mounting seat 702, and the output end of the servo motor 703 is connected to a connecting rod 704 with a threaded outer surface, and the connecting rod 704 runs through the interior of the mounting sleeve 705. The control rod 701 forms a lifting structure through the fixed block 404, which can control the movement of the information collection mechanism 8 to realize multi-directional data collection of the target product. The information collection mechanism 8 includes a gear 801 rotatably connected to the mounting sleeve 705, a rack 802 is arranged in front of the gear 801, a collection camera 803 is installed on the other end of the gear 801 connected to the mounting sleeve 705, the gear 801 forms a sliding structure behind the rack 802 through a connecting rod 704, and the rack 802 meshingly connected to the gear 801 can realize the rotation control of the gear 801 during the sliding process. The broadcasting mechanism 9 includes a wall-mounted body 901 fixedly connected to the mounting seat 702, and a speaker 902 is arranged on the left side of the wall-mounted body 901 to facilitate the real-time broadcast of voice commands.

[0110] The schemes in Example 1 and Example 3 are further introduced below in combination with specific working methods, as described below for details:

[0111] Specifically, when the intelligent detection system for voice interaction of intelligent terminals is used, the driving motor 301 installed inside the connecting block 203 is first started, and the rotation of the walking wheel 303 is controlled by the output shaft 302 to make the device run to the selected position. In the process of the device walking, when encountering a small obstacle or a bump on the ground, multiple groups of mounting rods 202 operate synchronously and cooperatively, so that the walking wheel 303 on the obstacle moves upward alone, and the state of the other walking wheels 303 remains unchanged. Compared with traditional walking, the design and use of this structure can make the device have an obstacle avoidance function, avoid high-intensity vibration from causing damage to precision instruments, and after crossing the obstacle, under the action of the strong spring 206, the first connecting rod 204 can slide and reset inside the second connecting rod 205, so that the walking wheel 303 is reset, which can ensure the stable operation of the device.

[0112] After the device moves to the selected position, the servo motor 402 installed on the right side of the installation box 401 is started, so that the transmission rod 403 drives the fixed block 404 to rotate clockwise. Since the fixed block 404 is meshed and connected with the control rod 701, the control rod 701 can drive the mounting seat 702 to rise to the selected position, which is convenient for the detection of the target product. The transmission rod 403 can be limited by the engagement of the vertical rod 601 and the stop block 405 to prevent it from rotating under pressure, thereby ensuring the stability of the device during the detection process. At the same time, when the transmission rod 403 rotates counterclockwise and drives the control rod 701 to move downward, the second movable block 504 meshed with the stop block 405 can be used to make the movable rod 501 drive the extrusion block 505 to move forward and squeeze the vertical rod 601, so that it rotates around the fixed rod 602 as the center and disengages from the stop block 405, which is convenient for the downward movement and resetting of the lifting mechanism 7.

[0113] The voice command can be broadcasted through the speaker 902 installed on the wall-mounted body 901. During the detection of the target product, the servo motor 703 installed on the left side of the mounting seat 702 can be started. During the rotation of the connecting rod 704, the mounting sleeve 705 can slide in the internal groove of the mounting seat 702, and at the same time drive and rotate the gear 801 and the acquisition camera 803 connected to the top of the mounting sleeve 705 to slide. Since the gear 801 is meshed with the rack 802, during the sliding process of the gear 801, the gear 801 can drive the acquisition camera 803 to rotate under the action of the rack 802, so that the focus of the acquisition camera 803 is always on the detection product, so as to achieve the purpose of multi-directional information data scanning of the selected product, make the detection result more accurate, and enhance the practicality of the device.

[0114] During the test, the background system converts the voice text information to be tested into audio, and plays the audio information to be tested through the speaker 902. At the same time, the camera 803 records the feedback of the intelligent device on each instruction, and inputs the acquired video or photo into the text recognition module and the target detection module respectively, wherein the text recognition module includes text detection and text recognition, namely: the relevant coordinates of the text area are obtained by text detection and the text coordinate information is transmitted to the text recognition algorithm for recognition to obtain text information. The target detection is performed by yolov5 to mark the target in the video or photo, and record the text category information. The voice test instruction is subjected to natural language processing to extract keywords, and matched with text recognition or target detection. If the match is successful, it is marked as successful, otherwise it is failed. After all tests are completed, a test report is generated.

[0115] Embodiment 4:

[0116] The solution in Example 2 is further introduced below in combination with a specific working method, as described below:

[0117] like Figure 12-Figure 19 As shown, this embodiment proposes an intelligent detection system for voice interaction of a smart terminal, including a walking mechanism 11. The walking mechanism 11 is provided to realize the movement of the device. A lifting mechanism 21 for adjusting the height is arranged on the top of the walking mechanism 11. The lifting mechanism 21 is provided to realize the height adjustment of the device. A labeling mechanism 31 for marking is arranged on the top of the lifting mechanism 21. The labeling mechanism 31 is provided to mark unqualified smart devices, which is convenient for distinguishing during transportation. An image acquisition device 41 for recording video clips is arranged on the top of the labeling mechanism 31. The image acquisition device 41 is provided to capture videos and photos. Audio devices 51 for playing voice commands are arranged on both sides of the labeling mechanism 31. The audio devices 51 are provided to play and receive voice.

[0118] As a preferred embodiment, on the basis of the above-mentioned method, the walking mechanism 11 further includes a fixed plate 111, and the four corners of the bottom end of the fixed plate 111 are fixedly connected with universal wheels 113, and laser radars 112 are embedded and fixed around the fixed plate 111. The laser radars 112 can prevent the device from colliding with unknown objects when moving, causing damage to the equipment, thereby improving the practicability of the product.

[0119] As a preferred embodiment, on the basis of the above-mentioned manner, the lifting mechanism 21 further includes a lifting frame 211 fixedly connected to the fixed plate 111, a lifting slot is provided at the top of the lifting frame 211, a lifting slot is slidably connected to a lifting column 212, a control box 61 is fixedly connected to the top of the lifting column 212, a placement port is provided at the front of the control box 61, a rack 213 is fixedly provided on one side of the lifting column 212, a gear shaft 214 is rotatably connected to one side of the rack 213 is provided in the lifting frame 211, a first stepper motor 215 is fixedly provided at the bottom of the gear shaft 214 in the lifting frame 211, the first stepper motor 215 is meshed with the gear shaft 214 through a gear, and the rotation of the first stepper motor 215 is realized to drive the rotation of the gear shaft 214, and the gear shaft 214 is meshed with the rack 213 to control the lifting and lowering of the lifting column 212, thereby adjusting the height of the control box 61 and improving the practicability of the device.

[0120] As a preferred embodiment, on the basis of the above-mentioned method, further, the labeling mechanism 31 includes a labeling box 311 slidably connected to the placement port, the labeling box 311 is internally rotatably connected to two winding disks 312, the labeling box 311 is rotatably connected to a guide shaft 313 located on one side of the winding disk 312, and the winding disk 312 is provided to realize the winding and storage of the label tape, the labeling box 311 is rotatably connected to a labeling column 314 located on one side of the guide shaft 313, the top and bottom of the labeling column 314 are provided with placement grooves, the placement groove is rotatably connected to a rotating roller 315, and the rotating roller 315 is provided to facilitate the movement of the label tape on the labeling column 314, thereby improving the practicability of the device, and a slide plate 611 is fixedly provided in the control box 61, and a slide groove is provided on the top of the slide plate 611. A first electric telescopic rod 612 located on one side of the slide is fixedly installed on 611, and a motor plate 613 is slidably connected inside the slide, and one side of the motor plate 613 is fixedly connected to the telescopic end of the first electric telescopic rod 612, and a third stepper motor 614 is provided at the top of the motor plate 613, and a second stepper motor 615 located on one side of the third stepper motor 614 is provided on the motor plate 613, and bevel gears are fixedly provided at the top of the rotating shafts of the third stepper motor 614 and the second stepper motor 615, and the third stepper motor 614 and the second stepper motor 615 are respectively engaged with the winding disk 312 and the labeling column 314, and the movement of the label tape is controlled by the rotation of the third stepper motor 614, and the angle adjustment of the labeling column 314 is realized by the rotation of the second stepper motor 615, thereby improving the practicability of the equipment.

[0121] As a preferred embodiment, on the basis of the above-mentioned manner, further, the image acquisition device 41 includes a fourth stepper motor 411 fixedly connected to the control box 61, the top end of the rotating shaft of the fourth stepper motor 411 is fixedly connected to a connecting plate 412, the top end of the connecting plate 412 is fixedly connected to a connecting seat 413, and the top of the connecting seat 413 is fixedly connected to a camera 414. Through the provided camera 414, the acquisition of test videos or photos is realized, and through the provided fourth stepper motor 411, the direction of the camera 414 is adjusted, and a second electric telescopic rod 415 located on one side of the connecting seat 413 is fixedly provided on the connecting plate 412, and the second electric telescopic rod 415 is rotatably connected to the camera 414. By controlling the extension and retraction of the second electric telescopic rod 415, the angle of the camera 414 is adjusted.

[0122] As a preferred embodiment, on the basis of the above-mentioned manner, further, the audio device 51 includes a third electric telescopic rod 511 rotatably connected to the control box 61, the telescopic end of the third electric telescopic rod 511 is rotatably connected to the wall 512, and the wall 512 is fixedly connected to the speaker 513, and the connection between the third electric telescopic rod 511 and the wall 512 is rotatably connected to a fifth stepper motor 514, and the wall 512 is driven to rotate by the fifth stepper motor 514, so as to adjust the playing direction of the speaker 513, and the side of the third electric telescopic rod 511 is rotatably connected to a sixth stepper motor, and the sixth stepper motor is used to drive the third electric telescopic rod 511 to rotate by the third electric telescopic rod 511 and the sixth stepper motor, so as to adjust the position angle of the wall 512, thereby adjusting the position of the speaker 513, thereby improving the practicability of the equipment.

[0123] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

[0124] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solution of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, a person of ordinary skill in the art can also make many forms of specific changes under the guidance of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A detection system for a smart terminal voice interaction detection method, the detection method comprising the following steps: S101. The tester formulates the test task through the test system platform; S102. After the intelligent robot autonomously navigates to the selected position according to the test task, it adjusts the corresponding operation and observation height through its own motion components; S103. The intelligent robot plays voice commands through the speakers to control the intelligent equipment, judges the intelligent control effect, and the camera records the corresponding video or photos to compare the control effect; S104. The acquired video or photo is input into the corresponding text recognition algorithm or target detection algorithm; S105. The recognized text target and the voice command are matched to obtain the test result and generate a test report; The detection system includes a carrier board (1), characterized in that, The lower surface of the carrying plate (1) is provided with an obstacle avoidance mechanism (2) for obstacle avoidance and shock reduction, and both ends of the obstacle avoidance mechanism (2) are provided with walking mechanisms (3). The upper surface of the carrying plate (1) is provided with an adjustment mechanism (4), and an anti-falling mechanism (5) for anti-falling protection is installed in the adjustment mechanism (4), and the anti-falling mechanism (5) is connected to a limiting mechanism (6). The top of the adjustment mechanism (4) is provided with a lifting mechanism (7), and the top of the lifting mechanism (7) is provided with an information collection mechanism (8), and the information collection mechanism (8) is provided with a lifting mechanism (7). A broadcasting mechanism (9) is arranged on one side of the structure (8); the obstacle avoidance mechanism (2) comprises a connecting base (201) fixed to the carrying plate (1); two groups of mounting rods (202) are symmetrically arranged at the front and rear ends of the connecting base (201); each group of mounting rods (202) is provided with two mounting rods; a connecting block (203) is arranged at the other end of the mounting rod (202) connected to the connecting base (201); the cross section of the mounting rod (202) is an "H"-shaped structure; the inner side of the mounting rod (202) is connected to the first connecting rod (204) One end of the first connecting rod (204) is hinged, and the other end of the first connecting rod (204) is provided with a second connecting rod (205), the first connecting rod (204) and the second connecting rod (205) are connected by a strong spring (206), and the second connecting rod (205) is connected to the connecting base (201); the walking mechanism (3) includes a driving motor (301) fixed to the connecting block (203), the output end of the driving motor (301) is connected to an output shaft (302), and the end of the output shaft (302) is provided with a walking wheel (303), and the adjustment mechanism ( 4) comprising an installation box (401) fixed to the carrier plate (1), a servo motor (402) being installed on one side above the installation box (401), an output end of the servo motor (402) being connected to a transmission rod (403) extending into the interior of the installation box (401), a fixing block (404) being arranged at both left and right ends of the transmission rod (403), the fixing block (404) being arranged symmetrically about the central axis of the stop block (405), and the outer surfaces of the fixing block (404) and the stop block (405) both being in a meshing structure; A support platform is provided inside the installation box (401), and the anti-fall mechanism (5) includes a movable rod (501) slidably connected to the support platform. Two grooves are provided inside the movable rod (501), and a first movable block (502) and a second movable block (504) are respectively provided in the two grooves. The movable rod (501) and the first movable block (502) and the movable rod (501) and the second movable block (504) are connected by a first spring (503), and the first movable block (502) and the second movable block (504) are arranged opposite to each other. An extrusion block (505) is fixed on one side of the top of the movable rod (501), and the first movable block (502) and the second movable block (504) both form a sliding structure inside the movable rod (501). The block (502) and the second movable block (504) are both meshed and connected with the stop block (405); the limiting mechanism (6) comprises a vertical rod (601) which is engaged and connected with the stop block (405); a fixing rod (602) is passed through the interior of the vertical rod (601); the vertical rod (601) can rotate around the fixing rod (602); a support plate (603) is provided at the other end of the fixing rod (602) connected to the vertical rod (601); a spring connecting plate is also installed on the support plate (603); a second spring (604) is fixed on the spring connecting plate; a clamping block which is engaged with the stop block (405) is provided on one side of the upper part of the vertical rod (601); and the other side is in contact with the second spring (604); the rotation angle of the lower part of the vertical rod (601) is limited by the extrusion block (505); The lifting mechanism (7) comprises a control rod (701) meshedly connected to a fixing block (404); a mounting seat (702) is fixed to the top of the control rod (701); a servo motor (703) is mounted on one end of the mounting seat (702); an output end of the servo motor (703) is connected to a connecting rod (704) having a threaded outer surface; the connecting rod (704) passes through the interior of a mounting sleeve (705); the control rod (701) forms a lifting structure through the fixing block (404); The collection mechanism (8) comprises a gear (801) rotatably connected to the mounting sleeve (705), a rack (802) is arranged in front of the gear (801), a collection camera (803) is installed on the top of the gear (801), the gear (801) forms a sliding structure on one side of the rack (802) through a connecting rod (704), and the broadcasting mechanism (9) comprises a wall-mounted body (901) fixedly connected to the mounting seat (702), and a speaker (902) is arranged on one side of the wall-mounted body (901).

Images