Vehicle machine function triggering interaction test system and method based on video identification

Abstract

The invention discloses a vehicle function triggering interaction test system and method based on video recognition, a robot module is electrically connected with an upper computer module, a hardware execution unit is electrically connected with an interaction recognition unit and an execution control unit, and a test management unit is electrically connected with the execution control unit. A hardware execution unit identifies a vehicle machine screen of an upper computer module to obtain specific position parameters, then a test management unit selects a test task, and an execution control unit configures all test actions; the test button is clicked, the execution control unit receives a test action, analyzes the test action and converts the test action into a physical instruction, the hardware execution unit executes a test and captures key parameters and performs data acquisition in the test process, and the upper computer module receives data, processes and analyzes the data and generates a corresponding test report. Through automatic testing and triggering of the multi-modal scene, manual intervention is reduced, the testing time is shortened, and the accuracy of key indexes is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, and more particularly to a vehicle machine function triggering interaction test system and method based on video recognition. BACKGROUND

[0002] Nowadays, new energy vehicles are evolving towards intelligent travel terminals at an explosive speed. Intelligent cockpits, as the core carriers of human-computer interaction, have become the strategic highlands of differentiation competition for vehicle manufacturers. The intelligent experience, especially the fluency, safety and scene adaptability of vehicle machine interaction, is affecting the user experience. The test of the vehicle machine interaction screen is particularly important, and is the core link of guaranteeing user experience, protecting driving safety, supporting brand competition and promoting technological progress, and is related to the realization of the intelligent value of new energy vehicles.

[0003] In the prior art, part of the vehicle machine system test scheme includes: using a single actuator to execute a certain vehicle machine function operation, such as a simulated finger actuator to verify a touch function. And through a fixed script, the single interaction scene is automatically verified, such as repeatedly verifying the start and exit of a single function of the software. With the help of a basic data acquisition tool, vehicle machine operation parameters are recorded, such as collecting software execution response time.

[0004] However, the above prior art has the following defects: 1. Only single function repeated test can be realized, which cannot meet the vehicle machine interaction test verification under the existing multi-modal scene. 2. Manual intervention is still needed most of the time, the test automation degree is low, and the test personnel's energy is wasted. 3. The management and processing of test data are still cumbersome.

[0005] Therefore, how to provide a vehicle machine function triggering interaction test system and method based on video recognition has become a technical problem to be solved in the field. SUMMARY

[0006] The purpose of the present application is to provide a new technical solution of a vehicle machine function triggering interaction test system and method based on video recognition.

[0007] According to a first aspect of the present application, a vehicle machine function triggering interaction test system based on video recognition is provided, comprising: a robot module and an upper computer module;

[0008] The robot module is electrically connected with the upper computer module, the robot module is provided with a hardware execution unit, an interaction recognition unit and an execution control unit, the hardware execution unit is electrically connected with the interaction recognition unit and the execution control unit respectively, the interaction recognition unit is used for capturing screen display information, touch operation trajectory information and gesture action information of the upper computer module in real time, and the execution control unit is used for receiving a test action issued by the upper computer module and outputting a value to the hardware execution unit.

[0009] The host computer module comprises a test management unit and an execution control unit, the test management unit is electrically connected with the execution control unit, the test management unit is used for formulating test tasks and configuring test actions, and the execution control unit is used for executing parameter configuration of the test actions and conveying to the robot module.

[0010] Optionally, the robot module further comprises a data acquisition communication unit, the data acquisition communication unit is connected with the hardware execution unit, the interaction recognition unit and the execution control unit, and the data acquisition communication unit is used for acquiring and storing original data in a test process and transmitting to the host computer module.

[0011] Optionally, the hardware execution unit comprises a 4-DOF mechanical arm, the maximum working radius of the 4-DOF mechanical arm is 580 mm, and the repeat positioning accuracy is ±0.05 mm.

[0012] Optionally, the hardware execution unit further comprises a high-definition camera, a voice player and a voice receiver, the high-definition camera is used for shooting gesture actions, the voice receiver is used for receiving voice information, and the voice player is used for playing sound.

[0013] Optionally, the host computer module further comprises a data management unit, the data management unit is electrically connected with the test management unit, the execution control unit and the data acquisition communication unit, the data management unit is used for receiving data information uploaded by the robot module, processing the data information, and generating a corresponding test report according to test requirements.

[0014] According to a second aspect of the present application, a vehicle machine function trigger interaction test method based on video recognition is provided, which adopts the vehicle machine function trigger interaction test system based on video recognition, and the test method comprises the following steps:

[0015] The hardware execution unit recognizes a vehicle machine screen of the host computer module and acquires specific position parameters;

[0016] A test task is selected in the host computer module;

[0017] Based on the selected test task, the host computer module configures all test actions;

[0018] A test button of a control panel of the host computer module is executed, the robot module receives and analyzes the test actions, converts the test actions into physical instructions, and executes the test;

[0019] Key parameters are captured by the hardware execution unit, and data acquisition is performed;

[0020] The data information is transmitted to the host computer module, the host computer module performs data processing and analysis, and a corresponding test report is generated.

[0021] The beneficial effects of the present application are:

[0022] The robot module is electrically connected with the host computer module, the hardware execution unit is electrically connected with the interaction recognition unit and the execution control unit respectively, and the test management unit is electrically connected with the execution control unit. In the test process, the hardware execution unit recognizes the car machine screen of the host computer module to obtain specific position parameters, then the test management unit of the host computer module selects a test task, and the execution control unit configures all test actions according to the test task; then, the test button in the control panel of the host computer module is clicked, the execution control unit of the robot module receives the test action for analysis, converts into a physical instruction, the hardware execution unit executes the test, and captures key parameters in the test process to collect data, the host computer module receives the data, performs data processing and analysis, and generates a corresponding test report. The present application reduces manual intervention, reduces test time, and improves the accuracy of key parameters through multi-modal scene automatic test and triggering.

[0023] Other features of the present application, and their advantages, will become apparent in the non-limiting detailed description of the application, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings incorporated in and forming a part of the specification illustrate embodiments of the present application and, together with the description, serve to explain the principles of the application.

[0025] Figure 1 is a structural diagram of the vehicle machine function trigger interaction test system based on video recognition of the present application;

[0026] Figure 2 is a flowchart of the vehicle machine function trigger interaction test method based on video recognition of the present application. DETAILED DESCRIPTION

[0027] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0028] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting to the scope of the application or its applications or uses.

[0029] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art(s) can not be discussed in detail herein. However, where appropriate, the techniques, methods, and apparatus should be considered as being part of the description of the application.

[0030] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Thus, other examples of the exemplary embodiments can have different values.

[0031] As Figure 1 shown, the embodiment of the application provides a vehicle-machine function trigger interaction test system based on video recognition, comprising a robot module and a host computer module.

[0032] The robot module is electrically connected with the host computer module, and the robot module is provided with a hardware execution unit, an interaction recognition unit and an execution control unit. The vehicle-machine function trigger robot is electrically connected with the hardware execution unit, the interaction recognition unit and the execution control unit. The interaction recognition unit is used for capturing the screen display information, the touch operation trajectory information and the gesture action information of the host computer module in real time. The execution control unit is used for receiving the test action issued by the host computer module and outputting the value to the hardware execution unit.

[0033] The host computer module comprises a test management unit and an execution control unit. The test management unit is electrically connected with the hardware execution unit, the interaction recognition unit and the execution control unit. The test management unit is used for formulating test tasks and configuring test actions. The execution control unit is used for performing parameter configuration of the test actions and delivering to the robot module.

[0034] Specifically, the data management unit can formulate test tasks and manage and classify the test tasks, and can also monitor the test task execution state in real time. In addition, the data management unit can configure test actions and form corresponding test cases according to the test tasks.

[0035] The test management unit can perform execution parameter configuration of the system, self-checking of the system function, and can also issue test cases and test actions to the hardware execution unit according to the test tasks. The test management unit dynamically controls the system start and stop according to the real-time state of the test management unit of the host computer module.

[0036] During the test process, the vehicle-machine screen of the host computer module is recognized by the hardware execution unit to obtain specific position parameters, and the vehicle-machine screen display content, touch operation trajectory and gesture action, etc. are captured in real time by the interaction recognition unit, and the host computer module function test provides the basis.

[0037] Then, the test management unit of the host computer module selects the test task, and the execution control unit configures all test actions according to the test task.

[0038] Then, click the test button in the control panel of the host computer module, the execution control unit of the robot module receives the test action for analysis, converts it into physical instructions, the hardware execution unit executes the test, and captures key parameters during the test to collect data, the host computer module receives the data, processes and analyzes the data, and generates a corresponding test report.

[0039] The robot module is electrically connected with the host computer module, the hardware execution unit is electrically connected with the interaction recognition unit and the execution control unit respectively, and the test management unit is electrically connected with the execution control unit. During the test, the hardware execution unit recognizes the car machine screen of the host computer module to obtain specific position parameters, then the test management unit of the host computer module selects a test task, and the execution control unit configures all test actions according to the test task, then the test button in the control panel of the host computer module is clicked, the execution control unit of the robot module receives the test action for analysis, converts it into physical instructions, the hardware execution unit executes the test, and captures key parameters during the test to collect data, the host computer module receives the data, processes and analyzes the data, and generates a corresponding test report. The application realizes automatic testing and triggering of multi-modal scenes, reduces manual intervention, reduces test time, and improves the accuracy of key parameters.

[0040] In an embodiment of the application, the robot module further comprises a data acquisition communication unit electrically connected with the hardware execution unit, the interaction recognition unit and the execution control unit. The data acquisition communication unit is used for collecting and storing original data in the test process, and transmitting to the host computer module.

[0041] The data acquisition communication unit is electrically connected with the hardware execution unit, the interaction recognition unit and the execution control unit, so that the data acquisition communication unit can collect and store original data during the test.

[0042] In addition, the data acquisition communication unit realizes device communication and data transmission function through TCP, CAN and other protocols.

[0043] In an embodiment of the application, the hardware execution unit comprises a 4-DOF mechanical arm with a maximum working radius of 580mm and a repeat positioning accuracy of ±0.05mm. Such setting can cover the full operation range of the car machine and the action is accurate.

[0044] In an embodiment of the vehicle-machine function trigger interaction test system based on video recognition of the application, the hardware execution unit further comprises a high-definition camera, a voice player and a voice receiver, the high-definition camera is used to shoot the gesture action of the mechanical arm, the voice receiver is used to receive voice information, and the voice player is used to play sound.

[0045] Specifically, the high-definition camera is electrically connected with the interaction recognition unit, the high-definition camera shoots the host computer module, screen position parameter information in the working process, and can also capture vehicle-machine screen display information, touch operation trajectory information and gesture action information in real time, and deliver the above information to the interaction recognition unit to provide a basis for vehicle-machine function test. The interaction recognition unit receives the above information and delivers it to the execution control unit.

[0046] The execution control unit receives the test action issued by the host computer module, combines the video shot by the high-definition camera and the video recognition technology of the interaction recognition unit, converts the test action into specific physical operation instructions, and makes the mechanical arm, voice player and other execution components of the driving hardware execution unit complete the actions of simulating finger touch, gesture operation and voice instruction, thereby realizing the test of various functions of the vehicle-machine.

[0047] In an embodiment of the vehicle-machine function trigger interaction test system based on video recognition of the application, the host computer module further comprises a data management unit, the data management unit is electrically connected with the test management unit, the execution control unit and the data acquisition communication unit, the data management unit is used to receive the data information uploaded by the robot module and process the data information, and generate a corresponding test report according to the test requirement.

[0048] Specifically, the data management unit receives the data uploaded from the hardware execution unit, performs automatic data processing, and generates a corresponding test report according to the test requirement.

[0049] In addition, the data management unit can also store past test data and reports, support multi-dimensional query, and compare the performance of vehicle-machines of different batches or versions.

[0050] According to the second aspect of the application, as Figure 2 shown, a vehicle-machine function trigger interaction test method based on video recognition is provided, which adopts the vehicle-machine function trigger interaction test system based on video recognition described above, and the test method comprises the following steps:

[0051] The hardware execution unit identifies the vehicle-machine screen of the host computer module and obtains specific position parameters;

[0052] Select a test task in the host computer module;

[0053] Based on the selected test task, the host computer module configures all test actions;

[0054] The test button of the control panel of the host computer module is executed, the robot module receives the test action for analysis, and the test is converted into a physical instruction and executed;

[0055] The key parameters are captured by the hardware execution unit, and data acquisition is performed;

[0056] The data information is transmitted to the host computer module, the host computer module performs data processing and analysis, and a corresponding test report is generated.

[0057] The application constructs a "perception-decision-execution" test method of a vehicle-machine function trigger interaction test system based on video recognition through the collaborative control of the high-precision mechanical arm, the interactive recognition unit, the execution control unit, the data acquisition communication unit, the test management unit, the execution control unit and the data management unit, and improves the automatic test and data management and processing capabilities.

[0058] Although some specific embodiments of the application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration, not for limiting the scope of the application. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the application. The scope of the application is defined by the appended claims.

Claims

1. A vehicle infotainment system function triggering interaction testing system based on video recognition, characterized in that, include: Robot module and host computer module; The robot module is electrically connected to the host computer module. The robot module is equipped with a hardware execution unit, an interaction recognition unit, and an execution control unit. The hardware execution unit is electrically connected to the interaction recognition unit and the execution control unit respectively. The interaction recognition unit is used to capture the screen display information, touch operation trajectory information, and gesture action information of the host computer module in real time. The execution control unit is used to receive the test actions issued by the host computer module and output the values ​​to the hardware execution unit. The host computer module includes a test management unit and an execution control unit. The test management unit is electrically connected to the execution control unit. The test management unit is used to formulate test tasks and configure test actions. The execution control unit is used to configure the parameters for executing test actions and transmit them to the robot module.

2. The in-vehicle infotainment system function triggering interaction test system based on video recognition according to claim 1, characterized in that, The robot module also includes a data acquisition and communication unit, which is connected to the hardware execution unit, the interaction recognition unit, and the execution control unit. The data acquisition and communication unit is used to collect and store raw data during the testing process and to transmit it to the host computer module.

3. The in-vehicle infotainment system function triggering interaction testing system based on video recognition according to claim 1, characterized in that, The hardware execution unit includes a 4-DOF robotic arm with a maximum working radius of 580mm and a repeatability of ±0.05mm.

4. The in-vehicle infotainment system function triggering interaction test system based on video recognition according to claim 2, characterized in that, The hardware execution unit also includes a high-definition camera, a voice player, and a voice receiver. The high-definition camera is used to capture hand gestures, the voice receiver is used to receive voice information, and the voice player is used to play sound.

5. The in-vehicle infotainment system function triggering interaction test system based on video recognition according to claim 2, characterized in that, The host computer module also includes a data management unit, which is electrically connected to the test management unit, the execution control unit, and the data acquisition and communication unit. The data management unit is used to receive data information uploaded by the robot module, process the data information, and generate corresponding test reports according to test requirements.

6. A method for testing in-vehicle infotainment system functions triggered by video recognition, characterized in that, The in-vehicle infotainment system based on video recognition for interactive testing, as described in any one of claims 1-5, comprises the following steps: The hardware execution unit identifies the vehicle's infotainment screen from the host computer module and obtains specific location parameters; Select the test task in the host computer module; Based on the selected test task, the host computer module configures all test actions; When the test button on the control panel of the host computer module is pressed, the robot module receives the test action, parses it, converts it into physical instructions, and executes the test. Key parameters are captured and data is collected through hardware execution units; The data is transmitted to the host computer module, which processes and analyzes the data and generates a corresponding test report.

Images