Vehicle automatic test method and system based on robot with body

Abstract

The invention relates to a vehicle automatic test method and system based on a body robot, and the method comprises the steps: obtaining a test task of the body robot, and the test task is a task for simulating the operation behavior of a user on a vehicle; decomposing the test task into an executable subtask sequence according to the test type and the environmental condition; and controlling the robot to execute the current subtask according to the priority of the subtask sequence, monitoring behavior actions of the vehicle when the robot executes the current subtask, and recording a task execution result of the robot. Therefore, the problems that a traditional automobile function testing method depends on manpower, is low in testing efficiency and hard to cover a dynamic scene, network testing is split in testing, and the man-machine cooperation efficiency is low are solved, the dependence of the testing on the manpower is reduced, the testing efficiency and the testing quality are improved, meanwhile, network-function integrated testing is guaranteed, and the testing efficiency is improved. Automatic testing of the automobile in a complex environment is realized by using the robot with the body based on vision, sound perception and information interaction cooperation.

Description

Technical Field

[0001] The present application relates to the field of robotics and intelligent vehicle technology, and in particular to a vehicle automation testing method and system based on an embodied robot. Background Art

[0002] With the rapid advancement of science and technology, the automotive industry is accelerating its transformation toward intelligent and connected vehicles. New features and systems, such as autonomous driving, smart cockpits, and connected vehicles, are constantly emerging. Technological evolution and policy breakthroughs are jointly propelling the industry into a new phase. In-vehicle systems are becoming increasingly complex, and cars are gradually evolving from mere wheels to intelligent brains that enable drivers and passengers to make safety-critical decisions. Furthermore, with intensifying competition in the global automotive market, automakers must continuously improve product quality and performance to meet consumer demand. Throughout the automotive industry's evolution, the potential impact of functional safety on drivers and passengers has become a critical concern, and ensuring the functional safety and stability of vehicles has become a major challenge.

[0003] Traditional testing methods include functional inspection, durability testing, safety testing, fuel economy testing, etc. These testing methods require testers to run the car for a long time in the laboratory to check whether the various components of the car are working properly; they need to simulate various unexpected situations, such as collisions, rollovers, etc., to check whether the car's safety performance is reliable; they need to simulate various driving conditions in the laboratory, such as high-speed driving, mountain driving, etc., to check whether the car's fuel economy performance is stable.

[0004] These testing methods are highly dependent on manual labor, requiring testers to operate on-site, which is inefficient and difficult to cover dynamic scenarios; network testing is fragmented, communication testing is separated from functional testing, and there is a lack of holistic verification; human-computer collaboration is inefficient and lacks intelligent interaction mechanisms, and the parsing and execution of test instructions are heavily dependent on manual intervention, which needs to be urgently addressed. Summary of the Invention

[0005] The present application provides a vehicle automation testing method and system based on an embodied robot to address the problems of traditional automobile functional safety testing methods, such as high manual dependence, low testing efficiency and difficulty in covering dynamic scenarios, network testing fragmentation during testing, and low efficiency of human-machine collaboration.

[0006] The first aspect of the present application provides a vehicle automation testing method based on an embodied robot, comprising the following steps: obtaining a test task for the embodied robot, wherein the test task is a task that simulates a user's operating behavior on a vehicle; decomposing the test task into an executable subtask sequence according to the test type and environmental conditions; controlling the embodied robot to execute the current subtask according to the priority of the subtask sequence, monitoring the vehicle's behavior while the embodied robot executes the current subtask, and recording the task execution results of the embodied robot.

[0007] Optionally, when the embodied robot monitors the behavior of the vehicle when performing the current subtask, it includes: judging whether the vehicle meets the preset abnormal conditions based on the behavior of the vehicle; if the vehicle meets the preset abnormal conditions, optimizing the action parameters of the embodied robot through the Lagrange multiplier method, and dynamically updating the test speed adjustment factor of the embodied robot based on the recursive least squares method.

[0008] Optionally, before the embodied robot performs the current subtask, it includes: using a preset reference signal to control the vehicle's onboard sensors and the embodied robot's sensors to perform spatiotemporal calibration, so as to collect multimodal feature data using the calibrated vehicle-mounted sensors and the embodied robot's sensors; performing feature fusion based on the multimodal feature data to obtain target features, so as to control the embodied robot to perform the current subtask according to the target features.

[0009] Optionally, decomposing the test task into an executable sub-task sequence according to the test type and environmental conditions includes: parsing the semantics of the test task using a pre-trained language model; and decomposing the test task into an executable sub-task sequence based on the semantics of the test task, the test type and the environmental conditions.

[0010] Optionally, when the embodied robot executes the current subtask, it includes: optimizing the test path of the current subtask according to a preset goal of minimizing the total time of the test task.

[0011] Optionally, when judging whether the vehicle meets the preset abnormal conditions based on the behavior of the vehicle, it includes: recording the interaction log between the vehicle and the embodied robot to judge whether the vehicle meets the preset abnormal conditions based on the behavior of the vehicle in the interaction log.

[0012] Optionally, when recording the task execution results of the embodied robot, it includes: collecting the braking distance, energy consumption and response time of the embodied robot, and collecting the task completion, efficiency and safety of the embodied robot after executing the current subtask; generating the task execution results of the embodied robot based on the braking distance, energy consumption and response of the embodied robot, and the task completion, efficiency and safety of the embodied robot, and generating a test report based on the task execution results and uploading it to a preset terminal; optimizing the test task according to the task execution results.

[0013] Optionally, using a preset reference signal to control the onboard sensors of the vehicle and the sensors of the embodied robot to perform spatiotemporal calibration includes: using a preset time calibration method to control the onboard sensors of the vehicle and the sensors of the embodied robot to perform spatiotemporal calibration, wherein the preset time calibration method is:

[0014]

[0015] Among them, t calibrated is the time after calibration, f sensor is the sensor sampling frequency, t base is the preset reference signal, t sensor is the sensor sampling time.

[0016] Optionally, the feature fusion method is:

[0017] F fused =αF visual +βF audio +γF text ;

[0018] Among them, α, β, and γ are weight coefficients, and F visual is the visual modality feature, F audio is the auditory modality feature, F text It is the text modality feature.

[0019] The second aspect of the present application provides a vehicle automation testing system based on an embodied robot, including: a task acquisition module for acquiring a test task for the embodied robot, wherein the test task is a task that simulates the user's operating behavior on the vehicle; a task decomposition module for decomposing the test task into an executable subtask sequence according to the test type and environmental conditions; and a task execution module for controlling the embodied robot to execute the current subtask according to the priority of the subtask sequence, monitoring the vehicle's behavior when the embodied robot executes the current subtask, and recording the task execution results of the embodied robot.

[0020] Optionally, when the embodied robot monitors the behavior of the vehicle when performing the current subtask, the task execution module is used to: determine whether the vehicle meets the preset abnormal conditions based on the vehicle's behavior; if the vehicle meets the preset abnormal conditions, optimize the action parameters of the embodied robot through the Lagrange multiplier method, and dynamically update the test speed adjustment factor of the embodied robot based on the recursive least squares method.

[0021] Optionally, before the embodied robot performs the current subtask, the task execution module is used to: use a preset reference signal to control the vehicle's onboard sensors and the embodied robot's sensors to perform spatiotemporal calibration, so as to collect multimodal feature data using the calibrated onboard sensors and the embodied robot's sensors; perform feature fusion based on the multimodal feature data to obtain target features, so as to control the embodied robot to perform the current subtask according to the target features.

[0022] Optionally, the task decomposition module is used to: parse the semantics of the test task using a pre-trained language model; and decompose the test task into an executable subtask sequence based on the semantics of the test task, the test type and the environmental conditions.

[0023] Optionally, when the embodied robot executes the current subtask, the task execution module is used to: optimize the test path of the current subtask according to a preset goal of minimizing the total time of the test task.

[0024] Optionally, when judging whether the vehicle meets the preset abnormal conditions based on the vehicle's behavior, the task execution module is used to: record the interaction log between the vehicle and the embodied robot to judge whether the vehicle meets the preset abnormal conditions based on the vehicle's behavior in the interaction log.

[0025] Optionally, when recording the task execution results of the embodied robot, the task execution module is used to: collect the braking distance, energy consumption and response time of the embodied robot, and collect the task completion, efficiency and safety of the embodied robot after executing the current subtask; generate the task execution results of the embodied robot based on the braking distance, energy consumption and response of the embodied robot, and the task completion, efficiency and safety of the embodied robot, and generate a test report based on the task execution results and upload it to a preset terminal; optimize the test task according to the task execution results.

[0026] Optionally, the task execution module is configured to: control the onboard sensors of the vehicle and the sensors of the embodied robot to perform spatiotemporal calibration using a preset time calibration method, wherein the preset time calibration method is:

[0027]

[0028] Among them, t calibrated is the time after calibration, f sensor is the sensor sampling frequency, t base is the preset reference signal, t sensor is the sensor sampling time.

[0029] Optionally, the feature fusion method is:

[0030] F fused =αF visual +βF audio +γF text ;

[0031] Among them, α, β, and γ are weight coefficients, and F visual is the visual modality feature, F audio is the auditory modality feature, F text It is the text modality feature.

[0032] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle automated testing method based on the embodied robot as described in the above embodiment.

[0033] A fourth aspect of the present application provides a computer program product having a computer program stored thereon, which is executed by a processor to implement the vehicle automation testing method based on an embodied robot as described in the above embodiment.

[0034] In the above implementation, a test task for the embodied robot is obtained, which simulates a user's vehicle operation. The test task is then broken down into a sequence of executable subtasks based on the test type and environmental conditions. The embodied robot is controlled to execute the current subtask based on the priority of the subtask sequence. The vehicle's behavior is monitored while the embodied robot executes the current subtask, and the results of the embodied robot's task execution are recorded. This solves the problems of traditional automotive functional safety testing methods, such as high manual dependency, low test efficiency, difficulty covering dynamic scenarios, fragmented network testing, and low human-machine collaboration efficiency. It reduces testing reliance on manual labor, improves test efficiency and quality, and ensures integrated network-functional testing. Embodied robots, based on visual and acoustic perception and information interaction and collaboration, enable automated testing of vehicle functions and performance in complex environments.

[0035] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0037] Figure 1 A flowchart of a vehicle automated testing method based on an embodied robot according to an embodiment of the present application;

[0038] Figure 2 Flowchart of a vehicle automated testing method based on an embodied robot according to one embodiment of the present application;

[0039] Figure 3 is an example diagram of a vehicle automated testing system based on an embodied robot according to an embodiment of the present application;

[0040] Figure 4 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0042] The following describes an embodiment of the present invention's vehicle automated testing method and system based on an embodied robot with reference to the accompanying drawings. In response to the problems mentioned in the background art above, such as the high manual dependency, low test efficiency, difficulty in covering dynamic scenarios, network testing fragmentation during testing, and low human-machine collaboration efficiency, the present invention provides a vehicle automated testing method based on an embodied robot. In this method, a test task for the embodied robot is obtained, which is a task that simulates a user's operating behavior on a vehicle; the test task is decomposed into an executable subtask sequence based on the test type and environmental conditions; the embodied robot is controlled to execute the current subtask based on the priority of the subtask sequence; the vehicle's behavior is monitored while the embodied robot executes the current subtask, and the task execution results of the embodied robot are recorded. Thus, the present invention solves the problems of the high manual dependency, low test efficiency, difficulty in covering dynamic scenarios, network testing fragmentation during testing, and low human-machine collaboration efficiency of the traditional functional safety testing method for automobiles. The present invention reduces the dependence on manual labor, improves test efficiency and test quality, and ensures network-function integrated testing. The embodied robot is used to achieve automated testing of automobile functions and performance in complex environments based on visual, sound perception, and information interaction.

[0043] Specifically, Figure 1 A schematic flow chart of a vehicle automated testing method based on an embodied robot provided in an embodiment of the present application.

[0044] like Figure 1 As shown, the vehicle automation testing method based on the embodied robot includes the following steps:

[0045] In step S101 , a test task of the embodied robot is obtained, where the test task is a task simulating a user's operation behavior on a vehicle.

[0046] Specifically, the content of the embodied robot's test tasks is determined based on basic operations, driving behavior, human-computer interaction, complex scenarios, safety and emergency response, simulating the user's operating actions and behaviors on the vehicle from different perspectives, as well as the expected expected results and important information about the vehicle status.

[0047] In step S102 , the test task is decomposed into executable subtask sequences according to the test type and environmental conditions.

[0048] Optionally, in some embodiments, the test task is decomposed into an executable sub-task sequence according to the test type and environmental conditions, including: using a pre-trained language model to parse the semantics of the test task; and decomposing the test task into an executable sub-task sequence based on the semantics of the test task, the test type and environmental conditions.

[0049] The embodied robot receives the test tasks set by the user, parses the task execution steps according to the test tasks, uses pre-trained language models (such as BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer)) to parse the semantics of the test tasks, and decomposes the high-level test tasks into executable sub-task sequences.

[0050] The subtask parameters are adjusted specifically according to the test type and environmental conditions of the test task. By combining natural language processing, environmental modeling and path planning technologies, they are decomposed into a subtask sequence that the embodied robot can understand and execute, and then the corresponding subtask sequence is executed in the vehicle environment.

[0051] After breaking down high-level test tasks into executable subtask sequences, the execution priority of the subtask sequences is set based on the importance or risk level of the functional modules, so that the test subtasks can be executed according to the task priority.

[0052] This technical solution leverages pre-trained language models to parse test tasks and their semantics, enabling a more accurate understanding of user intent and task requirements, reducing task execution errors caused by language ambiguity or misinterpretation. Based on the semantics, test type, and environmental conditions of the test task, high-level test tasks are dynamically decomposed into a sequence of executable subtasks, making the task decomposition process more flexible and adaptable to diverse testing scenarios and requirements.

[0053] In step S103, the embodied robot is controlled to execute the current subtask according to the priority of the subtask sequence, the behavior of the vehicle is monitored when the embodied robot executes the current subtask, and the task execution result of the embodied robot is recorded.

[0054] Optionally, in some embodiments, before the embodied robot performs the current subtask, it includes: using a preset reference signal to control the vehicle's onboard sensors and the embodied robot's sensors to perform spatiotemporal calibration, so as to collect multimodal feature data using the calibrated vehicle-mounted sensors and the embodied robot's sensors; performing feature fusion based on the multimodal feature data to obtain target features, so as to control the embodied robot to perform the current subtask according to the target features.

[0055] As you can see, the vehicle's operating data (speed, braking, energy consumption) and environmental parameters are collected in real time through onboard sensors, the robot's sensors (visible light and infrared cameras), and environmental sensors (temperature, humidity, light, noise, etc.). PPS (Pulse Per Second) signals are synchronized and spatially aligned with the multi-source data in microseconds. Through the collaborative design of hardware and software, high-precision sensor data fusion is achieved, providing comprehensive information for the embodied robot's perception and decision-making in complex environments.

[0056] Furthermore, in some embodiments, before the embodied robot performs the current subtask, it includes: using a preset reference signal to control the vehicle's onboard sensors and the embodied robot's sensors to perform spatiotemporal calibration, so as to collect multimodal feature data using the calibrated vehicle-mounted sensors and the embodied robot's sensors; performing feature fusion based on the multimodal feature data to obtain target features, so as to control the embodied robot to perform the current subtask according to the target features.

[0057] Optionally, in some embodiments, using a preset reference signal to control the spatiotemporal calibration of the vehicle's onboard sensors and the embodied robot's sensors includes: using a preset time calibration method to control the spatiotemporal calibration of the vehicle's onboard sensors and the embodied robot's sensors, wherein the preset time calibration method is:

[0058]

[0059] Among them, t calibrated is the time after calibration, f sensor is the sensor sampling frequency, t base is the preset reference signal, t sensor is the sensor sampling time.

[0060] In some embodiments, the feature fusion method is:

[0061] F fused =αF visual +βF audio +γF text

[0062] Among them, α, β, and γ are weight coefficients, and F visual is the visual modality feature, F audio is the auditory modality feature, F text It is the text modality feature.

[0063] In collaborative tasks involving embodied robots and vehicles, spatiotemporal alignment and multimodal feature fusion are key to ensuring efficient collaboration. The specific process is as follows: Using a PPS reference signal, linear interpolation is used to align timestamps for sensors with different sampling frequencies (such as radar and cameras).

[0064] Time synchronization calibration: Align the sensor clock with a preset reference signal (such as a hardware trigger pulse or NTP protocol) with an error of <1ms.

[0065] Spatial alignment calibration: Use a calibration plate (such as a chessboard) to calculate the extrinsic parameter matrix (rotation + translation) between sensors to achieve coordinate system transformation.

[0066] Furthermore, by collecting data information from various sensors, and then integrating visual, auditory, text and other modal features through weighted fusion (α, β, γ are weight coefficients) for different signals, the model's understanding ability is improved.

[0067] Specifically, within a calibrated spatiotemporal reference, the embodied robot's sensors collect multimodal feature data, including point cloud features (e.g., obstacle distance and volume, using PointNet++); visual features (e.g., cargo color and texture, using YOLOv8); and IMU features (e.g., robot posture and acceleration). Feature fusion dynamically adjusts the weights of each modal feature to generate a target feature. Based on this target feature, the embodied robot generates control instructions (e.g., motion path, gripping force) to execute the current subtask.

[0068] This technical solution utilizes preset reference signals to perform spatiotemporal calibration between the vehicle's onboard sensors and the embodied robot's sensors. This ensures temporal and spatial alignment of sensor data, eliminating data deviations caused by factors such as sensor position and time delays. This improves the accuracy and consistency of sensor data. Feature fusion based on multimodal feature data effectively integrates data from different modalities to extract more representative and discriminative target features. These fused features more accurately reflect the true state of the vehicle and robot's surroundings, providing a more reliable basis for subsequent task execution.

[0069] Optionally, in some embodiments, when determining whether a vehicle meets a preset abnormal condition based on the vehicle's behavior, it includes: recording an interaction log between the vehicle and the embodied robot to determine whether the vehicle meets the preset abnormal condition based on the vehicle's behavior in the interaction log.

[0070] In collaborative tasks between embodied robots and vehicles, abnormal vehicle behavior (such as sudden braking, sudden steering, and sensor failure) may lead to mission failure or safety accidents. By recording interaction logs and analyzing vehicle behavior, the embodied robot can determine in real time whether the vehicle meets the preset abnormal conditions. The specific process is as follows:

[0071] Interaction logging:

[0072] Record vehicle status: speed, acceleration, steering angle, brake signals, sensor data (such as lidar point cloud, camera images).

[0073] Record the state of the embodied robot: position, posture, task execution phase, and sensor data (such as IMU and force sensors).

[0074] Recording method:

[0075] The vehicle and robot data are written to log files (such as CSV or JSON format) in real time through the robot operating system or a custom communication protocol (such as CAN bus + UDP).

[0076] The preset abnormal conditions may be:

[0077] Vehicle behavior indicators (such as acceleration and steering angle change rate) are calculated through a sliding window (such as the data of the last 1 second). If the indicator exceeds the threshold, it is determined that the vehicle meets the preset abnormal conditions.

[0078] Through the above technical solution, by recording the interaction log between the vehicle and the embodied robot, the various behavioral actions of the vehicle during the test process can be recorded in a comprehensive and detailed manner. Based on the vehicle behavioral actions in the interaction log, they can be compared with the preset abnormal conditions to accurately determine whether the vehicle is in an abnormal state. When the vehicle exhibits abnormal behavior, the time point and specific behavior of the problem can be quickly located by checking the interaction log, which helps testers quickly find the root cause of the problem and repair it.

[0079] Optionally, in some embodiments, when the embodied robot monitors the behavior of the vehicle while performing the current subtask, it includes: judging whether the vehicle meets the preset abnormal conditions based on the vehicle's behavior; if the vehicle meets the preset abnormal conditions, optimizing the action parameters of the embodied robot through the Lagrange multiplier method, and dynamically updating the test speed adjustment factor of the embodied robot based on the recursive least squares method.

[0080] Specifically, when the embodied robot is executing the current subtask, it starts recording video information, including the indicator lights, sound interactions and lights of the vehicle instruments, and monitors the behavior of the car in real time; the robot and the car are in the same local area network, and the robot interacts with the on-board network system, recording messages and logs in real time. By verifying the keywords in different test subtasks and capturing the keyword information, it can judge in real time whether the task execution results of different test subtasks meet expectations. For test subtasks that do not meet expectations, the log, message information, and video information are saved to facilitate subsequent manual investigation and analysis of the cause of the error.

[0081] During the test, the vehicle status and environmental information (such as road wetness and sudden changes in lighting) are monitored in real time. If the vehicle's response data exceeds the preset threshold or if the vehicle meets abnormal conditions, the embodied robot's action parameters are optimized through the Lagrange multiplier method, and the test speed adjustment factor is dynamically updated based on the recursive least squares method. The robot's action strategy can be quickly adjusted to adapt to abnormal situations, ensuring the smooth progress of the test and improving the flexibility and adaptability of the test.

[0082] Through the above technical solution, whether the vehicle meets the preset abnormal conditions is determined based on the vehicle's behavior and actions, and abnormal situations during the test process can be discovered and handled in a timely manner to avoid test failures or damage to test equipment.

[0083] Optionally, in some embodiments, when the embodied robot performs the current subtask, it includes: optimizing the test path of the current subtask according to a preset goal of minimizing the total time of the test task.

[0084] By optimizing the test path of the current subtask, the total test task time target is minimized. For example, in vehicle network testing, bandwidth allocation is optimized through state transition equations to ensure the real-time performance of high-priority sensor data.

[0085] dp[i][j]=max(dp[i-1][j-ck]+vk,dp[i-1][j])

[0086] Among them, dp[i][j] represents the maximum test value of the covered module when the first i test subtasks are executed and the remaining resources are j.

[0087] Through the above technical solution, by optimizing the test path and reducing unnecessary movement and waiting time, the embodied robot can complete the current subtask more efficiently, thereby shortening the total time of the entire test task.

[0088] Optionally, in some embodiments, when recording the task execution results of the embodied robot, it includes: collecting the braking distance, energy consumption and response time of the embodied robot, and collecting the task completion, efficiency and safety of the embodied robot after executing the current subtask; generating the task execution results of the embodied robot based on the braking distance, energy consumption and response of the embodied robot, and the task completion, efficiency and safety of the embodied robot, and generating a test report based on the task execution results and uploading it to a preset terminal; optimizing the test task according to the task execution results.

[0089] In collaborative tasks between embodied robots and vehicles, the system needs to comprehensively record the robot's key performance indicators (braking distance, energy consumption, response time) and task quality indicators (completion, efficiency, safety), and achieve iterative upgrades of task strategies by generating test reports and implementing reinforcement learning optimization. The specific process is as follows:

[0090] 1. Data collection and indicator definition

[0091] (1) Key performance indicators

[0092] Braking distance: Collection method: Use wheel odometer or lidar to record the trajectory and calculate the displacement difference before and after stopping.

[0093] Energy consumption: Collection method: The robot battery management system monitors current and voltage in real time and calculates energy consumption through integration.

[0094] Response time: Collection method: The difference between the timestamps of command sending and the start of robot movement is recorded by a high-precision timer.

[0095] (2) Task quality indicators

[0096] Task completion: Collection method: Confirm whether the task is completed through visual inspection or force sensor feedback.

[0097] Efficiency: Collection method: Statistics on task duration and number of completed tasks, and calculation of efficiency values.

[0098] Safety: Collection method: Comprehensive judgment through collision sensors, IMU attitude data and environmental monitoring (such as lidar obstacle distance).

[0099] 2. Task execution result generation

[0100] (1) Data integration

[0101] The collected performance indicators (braking distance, energy consumption, response time) and quality indicators (completion, efficiency, safety) are associated with the task records with the same timestamp, and the task execution results are generated through a weighted scoring model.

[0102] Further test report generation and upload

[0103] (1) Report content

[0104] Task overview: task ID, subtask type, and execution time.

[0105] Performance analysis: braking distance distribution diagram, energy consumption-time curve, response time histogram.

[0106] Quality assessment: completion heat map, efficiency comparison table, and security event log.

[0107] Comprehensive score: score of task execution results.

[0108] (2) Report generation and upload

[0109] Use Python (such as Matplotlib, Pandas) to generate visual reports and upload them to the preset terminal (such as cloud server or local monitoring platform) via HTTP / FTP protocol.

[0110] This application also optimizes the testing strategy:

[0111] (1) Reward mechanism design

[0112] The reward function accumulates rewards over a long period of time, thereby optimizing the test strategy (such as path planning and motion control parameters).

[0113] (2) Reinforcement Learning Model Iteration

[0114] Model selection: Use PPO (Proximal Policy Optimization) or SAC (Soft Actor-Critic) algorithm.

[0115] The embodied robot performs tasks in a simulation environment and records status, actions, and rewards.

[0116] The policy network parameters are updated through back-propagation, and the optimized test strategy is regularly deployed to real robot tests.

[0117] (3) Test strategy update

[0118] According to the output of the reinforcement learning model, the test task parameters (such as task priority and sensor sampling frequency) are dynamically adjusted.

[0119] Example update rule:

[0120] If the braking distance exceeds the threshold for three consecutive times, the upper limit of movement speed will be reduced by 10%.

[0121] If the energy consumption is too high, optimize the path planning algorithm to reduce redundant movements.

[0122] Through the above technical solution, by collecting the braking distance, energy consumption and response time of the embodied robot, and collecting the task completion, efficiency and safety of the embodied robot after performing the current subtask, the robot performance is accurately evaluated, a test report is generated, and the transparency and traceability of the test process are improved.

[0123] In order to enable those skilled in the art to further understand the vehicle automated testing method based on the embodied robot according to the embodiment of the present application, the following is a detailed description of the method with reference to specific embodiments. Figure 2 shown.

[0124] S1: Test task preparation

[0125] Determine the structure of the test tasks and simulate the user's operating actions and behaviors on the vehicle based on basic operations, driving behavior, human-computer interaction, complex scenarios, safety and emergency preparedness test tasks.

[0126] S2: Multimodal Data Acquisition and Synchronization

[0127] Real-time vehicle operating data and environmental parameters are collected through onboard sensors, robotic vision modules, and environmental sensors. Microsecond-level clock synchronization and spatial calibration of PPS signals are performed on multi-source data. Through the collaborative design of hardware and software, high-precision sensor data fusion is achieved, providing comprehensive information for the embodied robot's perception and decision-making in complex environments.

[0128] S3: Test task analysis and dynamic planning

[0129] The embodied robot receives test tasks issued by users, parses the task execution steps based on the test task, uses a pre-trained language model to analyze the test task's semantics, and decomposes the high-level test task into a sequence of executable subtasks. Subtask parameters are adjusted based on the test type and environmental conditions. By combining natural language processing, environmental modeling, and path planning technologies, the embodied robot efficiently parses the test task, generates executable subtasks, and then executes the subtasks in the vehicle environment.

[0130] S4: Intelligent Interaction:

[0131] According to the disassembled task subsequences, the embodied robot gradually performs the corresponding actions. While executing the tasks, it starts recording video information, instrument indicator lights, sound interactions, and lights, etc., to monitor the car's behavior in real time; the robot and the car are in the same local area network, and the robot interacts with the on-board network system, recording messages and logs in real time. Based on the verification keywords in the test task and the information of the keywords, it can judge in real time whether the operation results meet expectations. For operation cases that do not meet expectations, the log, message information, and video information are saved to facilitate subsequent manual investigation and analysis of the cause of the error.

[0132] S5: Intelligent Decision-Making

[0133] During the test, the vehicle status and environmental information are monitored in real time. If the vehicle's response data exceeds the preset threshold, the robot's action parameters are optimized using the Lagrange multiplier method, and the test speed adjustment factor α is dynamically updated based on the recursive least squares method to ensure test safety and effectiveness.

[0134] During the test, the execution result of the previous subtask is used to determine whether the next subtask needs to be continued, and the update rule is implemented through the state transition equation.

[0135] S6: Multimodal Feedback and Reinforcement Learning Optimization

[0136] After a round of testing, key metrics (braking distance, energy consumption, response time) are recorded and a test report is generated. The reinforcement learning model is optimized through a reward mechanism (task completion, efficiency, safety), and the testing strategy is iteratively updated.

[0137] Compared with the existing technology, the beneficial effects of this application are embodied in:

[0138] 1. Multimodal data fusion: Integrate visual, environmental, and vehicle data to improve the authenticity of test scenarios and the comprehensiveness of data.

[0139] 2. Dynamic adaptability: Adjust test parameters in real time to cope with complex environmental changes and ensure test accuracy.

[0140] 3. Intelligent interaction: Realize human-machine collaborative testing through natural language parsing and three-dimensional scene analysis.

[0141] 4. Autonomous learning optimization: Based on the feedback mechanism of reinforcement learning, continuously improve test efficiency and scenario coverage.

[0142] According to the embodied robot-based automated vehicle testing method proposed in the embodiments of the present application, a test task for the embodied robot is obtained, which is a task that simulates a user's operating behavior on a vehicle; the test task is decomposed into an executable subtask sequence based on the test type and environmental conditions; the embodied robot is controlled to execute the current subtask based on the priority of the subtask sequence, the vehicle's behavior is monitored while the embodied robot executes the current subtask, and the task execution results of the embodied robot are recorded. This solves the problems of traditional automotive functional safety testing methods, such as high manual dependence, low test efficiency and difficulty in covering dynamic scenarios, network testing fragmentation during testing, and low human-machine collaboration efficiency. It reduces the dependence on manual labor for testing, improves test efficiency and quality, and ensures network-function integrated testing. Embodied robots are used to achieve automated testing of automotive functions and performance in complex environments based on visual, sound perception, and information interaction and collaboration.

[0143] Next, an embodied robot-based vehicle automated testing system according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0144] Figure 3 4 is a block diagram of an embodied robot-based vehicle automated testing system according to an embodiment of the present application.

[0145] like Figure 3 As shown, the embodied robot-based vehicle automated testing system 10 includes: a task acquisition module 100 , a task decomposition module 200 and a task execution module 300 .

[0146] Among them, the task acquisition module 100 is used to obtain the test task of the embodied robot, which is a task that simulates the user's operating behavior on the vehicle; the task decomposition module 200 is used to decompose the test task into an executable sub-task sequence according to the test type and environmental conditions; the task execution module 300 is used to control the embodied robot to execute the current sub-task according to the priority of the sub-task sequence, monitor the vehicle's behavior when the embodied robot executes the current sub-task, and record the task execution results of the embodied robot.

[0147] Optionally, in some embodiments, when the embodied robot monitors the behavior of the vehicle when performing the current subtask, the task module 300 is executed to: determine whether the vehicle meets the preset abnormal conditions based on the vehicle's behavior; if the vehicle meets the preset abnormal conditions, optimize the action parameters of the embodied robot through the Lagrange multiplier method, and dynamically update the test speed adjustment factor of the embodied robot based on the recursive least squares method.

[0148] Optionally, in some embodiments, before the embodied robot performs the current subtask, the execution task module 300 is used to: use a preset reference signal to control the vehicle's onboard sensors and the embodied robot's sensors to perform spatiotemporal calibration, so as to use the calibrated onboard sensors and the embodied robot's sensors to collect multimodal feature data; perform feature fusion based on the multimodal feature data to obtain target features, so as to control the embodied robot to perform the current subtask according to the target features.

[0149] Optionally, in some embodiments, the task decomposition module 200 is used to: parse the semantics of the test task using a pre-trained language model; and decompose the test task into an executable subtask sequence based on the semantics, test type, and environmental conditions of the test task.

[0150] Optionally, in some embodiments, when the embodied robot performs the current subtask, the task module 300 is executed to optimize the test path of the current subtask according to a preset goal of minimizing the total time of the test task.

[0151] Optionally, in some embodiments, when determining whether a vehicle meets a preset abnormal condition based on the vehicle's behavior, the task module 300 is executed to: record an interaction log between the vehicle and the embodied robot to determine whether the vehicle meets the preset abnormal condition based on the vehicle's behavior in the interaction log.

[0152] Optionally, in some embodiments, when recording the task execution results of the embodied robot, the task module 300 is executed to: collect the braking distance, energy consumption and response time of the embodied robot, and collect the task completion, efficiency and safety of the embodied robot after executing the current subtask; generate the task execution results of the embodied robot based on the braking distance, energy consumption and response of the embodied robot, and the task completion, efficiency and safety of the embodied robot, and generate a test report based on the task execution results and upload it to a preset terminal; optimize the test task according to the task execution results.

[0153] Optionally, in some embodiments, the execution task module 300 is configured to: control the onboard sensors of the vehicle and the sensors of the embodied robot to perform spatiotemporal calibration using a preset time calibration method, wherein the preset time calibration method is:

[0154]

[0155] Among them, t calibrated is the time after calibration, f sensor is the sensor sampling frequency, t base is the preset reference signal, t sensor is the sensor sampling time.

[0156] Optionally, in some embodiments, the feature fusion method is:

[0157] F fused =αF visual +βF audio +γF text

[0158] Among them, α, β, and γ are weight coefficients, and F visual is the visual modality feature, F audio is the auditory modality feature, F text It is the text modality feature.

[0159] It should be noted that the aforementioned explanation of the embodiment of the vehicle automated testing method based on an embodied robot is also applicable to the vehicle automated testing system based on an embodied robot in this embodiment, and will not be repeated here.

[0160] According to the embodiment of the present application, a vehicle automation testing system based on an embodied robot is proposed, which obtains the test task of the embodied robot, which is a task that simulates the user's operating behavior on the vehicle; decomposes the test task into an executable sub-task sequence according to the test type and environmental conditions; controls the embodied robot to execute the current sub-task according to the priority of the sub-task sequence, monitors the vehicle's behavior when the embodied robot executes the current sub-task, and records the task execution result of the embodied robot. This solves the problems of traditional automobile functional safety testing methods with high manual dependence, low test efficiency and difficulty in covering dynamic scenarios, network testing separation during testing, and low human-computer collaboration efficiency, reduces the dependence of testing on manual labor, improves test efficiency and test quality, and ensures network-function integrated testing at the same time. Embodied robots are used based on vision, sound perception and information interaction and collaboration to realize automated testing of automobile functions and performance in complex environments.

[0161] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0162] Memory 401 , processor 402 , and computer programs stored in the memory 401 and executable on the processor 402 .

[0163] When the processor 402 executes the program, the vehicle automated testing method based on the embodied robot provided in the above embodiment is implemented.

[0164] Furthermore, the electronic device further includes:

[0165] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0166] The memory 401 is used to store computer programs that can be run on the processor 402 .

[0167] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0168] If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0169] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0170] The processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0171] An embodiment of the present application further provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned vehicle automated testing method based on an embodied robot.

[0172] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0173] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0174] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0175] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer program product for use with, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer program product" can be any device that can contain, store, communicate, propagate, or transmit a program for use with, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer program products include the following: an electrical connection having one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer program product may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or, if necessary, processing it in another suitable manner, and then storing it in a computer memory.

[0176] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0177] Those skilled in the art will understand that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer program product, which, when executed, includes one or a combination of the steps of the method embodiment.

[0178] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer program product.

[0179] The computer program product mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A vehicle automated testing method based on an embodied robot, characterized in that: The following steps are involved: Obtaining a test task for the embodied robot, wherein the test task is a task simulating a user's operation behavior of a vehicle; Decomposing the test task into executable subtask sequences according to the test type and environmental conditions; The embodied robot is controlled to execute the current subtask according to the priority of the subtask sequence, the behavior of the vehicle is monitored when the embodied robot executes the current subtask, and the task execution result of the embodied robot is recorded.

2. The method according to claim 1, characterized in that When the embodied robot monitors the behavior of the vehicle while performing the current subtask, the method includes: Determining whether the vehicle meets a preset abnormal condition based on the vehicle's behavior; If the vehicle meets the preset abnormal condition, the action parameters of the embodied robot are optimized by the Lagrange multiplier method, and the test speed adjustment factor of the embodied robot is dynamically updated based on the recursive least squares method.

3. The method according to claim 1, characterized in that Before the embodied robot performs the current subtask, the process includes: Using a preset reference signal to control the onboard sensors of the vehicle and the sensors of the embodied robot to perform spatiotemporal calibration, so as to collect multimodal feature data using the calibrated onboard sensors and the sensors of the embodied robot; Feature fusion is performed based on the multimodal feature data to obtain a target feature, so as to control the embodied robot to perform the current subtask according to the target feature.

4. The method according to claim 1, wherein Decomposing the test task into executable subtask sequences according to the test type and environmental conditions includes: Analyze the semantics of the test task using a pre-trained language model; The test task is decomposed into an executable subtask sequence based on the semantics of the test task, the test type and the environmental conditions.

5. The method according to claim 1, characterized in that When the embodied robot performs the current subtask, it includes: The test path of the current subtask is optimized according to a preset goal of minimizing the total test task time.

6. The method according to claim 2, characterized in that Before determining whether the vehicle meets a preset abnormal condition based on the vehicle's behavior, the method includes: The interaction log between the vehicle and the embodied robot is recorded to determine whether the vehicle meets a preset abnormal condition based on the behavior of the vehicle in the interaction log.

7. The method according to claim 1, characterized in that When recording the task execution results of the embodied robot, it includes: Collecting the braking distance, energy consumption, and response time of the embodied robot, and collecting the task completion, efficiency, and safety of the embodied robot after performing the current subtask; generating a task execution result of the embodied robot based on the braking distance, energy consumption, and response of the embodied robot, and the task completion, efficiency, and safety of the embodied robot, and generating a test report based on the task execution result and uploading it to a preset terminal; The test task is optimized according to the task execution result.

8. The method according to claim 3, characterized in that Using a preset reference signal to control the onboard sensor of the vehicle and the sensor of the embodied robot to perform spatiotemporal calibration, comprising: The vehicle's onboard sensors and the embodied robot's sensors are controlled to perform spatiotemporal calibration using a preset time calibration method, wherein the preset time calibration method is: Among them, t calibrated is the time after calibration, f sensor is the sensor sampling frequency, t base is the preset reference signal, t sensot is the sensor sampling time.

9. The method according to claim 3, characterized in that The feature fusion method is: F fused =αF visual +βF audio +γF text Among them, F fUsed is the target feature after fusion, α, β, γ are weight coefficients, F visual is the visual modality feature, F audio is the auditory modality feature, F text It is the text modality feature.

10. A vehicle automated testing system based on an embodied robot, characterized in that: include: A task acquisition module is used to acquire a test task for the embodied robot, wherein the test task is a task that simulates a user's operation behavior on the vehicle; A task decomposition module, configured to decompose the test task into executable subtask sequences according to the test type and environmental conditions; The task execution module is used to control the embodied robot to execute the current subtask according to the priority of the subtask sequence, monitor the behavior of the vehicle when the embodied robot executes the current subtask, and record the task execution results of the embodied robot.

Images