Intelligent driving system test method, test equipment and storage medium

Abstract

The invention relates to an intelligent driving system test method, test equipment and a storage medium, and the test method comprises the steps: determining a set value of each scene perception item in at least one scene perception item in a test scene according to the scene setting information of the test scene for a to-be-tested intelligent driving function; for each scene perception item, converting a set value of the scene perception item in the test scene into perception result data of the scene perception item corresponding to the test scene; and injecting all the sensing result data obtained by conversion into the intelligent driving controller, so that the intelligent driving controller performs motion control of the test vehicle based on the injected sensing result data. And the test vehicle completes the test of the intelligent driving function in the corresponding test scene based on the externally generated perception result data.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent driving test, and more specifically, to a method and apparatus for testing an intelligent driving system, and a computer-readable storage medium. Background Art

[0002] In order to ensure the application performance of an intelligent driving system, during the system development, a large number of on-road tests need to be carried out by a Vehicle Under Test (VUT), so as to determine whether the intelligent driving function meets the design expectations through the on-road tests. Herein, the VUT is a vehicle equipped with the intelligent driving system to be tested and evaluated according to a specific test procedure.

[0003] During the on-road tests, the staff needs to set up target objects such as dummy pedestrians and spliced vehicles in the test site according to the test scenario settings. When the intelligent driving function is not yet mature, the test vehicle is very likely to collide with the target objects during the test, which usually causes damage to parts of test equipment such as the test vehicle and target objects, communication interruption, etc. Furthermore, it takes a lot of time to troubleshoot problems, restore equipment or replace equipment, seriously affecting the test efficiency and also increasing the test cost.

[0004] In addition, during the system function development process, it is often necessary to perform multiple tests based on the same test scenario at the same or different development nodes, so as to evaluate the function development situation by synthesizing or comparing the results of multiple tests. However, in reality, it is very difficult to maintain a high degree of consistency of scenario elements in multiple tests, which affects the accuracy of test evaluation. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure propose a new technical solution for testing an intelligent driving system.

[0006] According to a first aspect of the present disclosure, there is provided a method for testing an intelligent driving system, the method including:

[0007] Determining a set value of each scene perception item in at least one scene perception item in the test scenario according to the scene setting information of the test scenario for the intelligent driving function to be tested;

[0008] For each scene perception item, converting the set value of the scene perception item in the test scenario into perception result data corresponding to the test scenario for the scene perception item; wherein, the perception result data corresponding to the scene perception item conforms to the perception result format requirement of the intelligent driving controller assembled in the test vehicle for the scene perception item, and the intelligent driving controller is a component in the intelligent driving system that executes the intelligent driving function;

[0009] Inject all the converted perception result data into the intelligent driving controller, so that the intelligent driving controller performs motion control of the test vehicle based on the injected perception result data.

[0010] Optionally, the injecting all the converted perception result data into the intelligent driving controller includes:

[0011] Pack the perception result data corresponding to all scene perception items into first scene perception result data according to the packaging structure requirements of the intelligent driving controller for the scene perception result;

[0012] Inject the first scene perception result data into the intelligent driving controller.

[0013] Optionally, the at least one scene perception item includes a first scene perception item. For each scene perception item, converting the set value of the scene perception item in the test scene into the perception result data corresponding to the test scene of the scene perception item includes:

[0014] For the first scene perception item, match the perception result data corresponding to the set value of the first scene perception item in the test scene in the perception result database of the first scene perception item as the perception result data corresponding to the test scene of the first scene perception item;

[0015] Wherein, the data format of the perception result data in the perception result database conforms to the perception result format requirements of the intelligent driving controller for the first scene perception item.

[0016] Optionally, the perception result data in the perception result database is extracted from pre-generated second scene perception result data, and the second scene perception result data is obtained through real-scene data collection.

[0017] Optionally, the second scene perception result data is obtained by the intelligent driving system through scene perception based on the real-scene data.

[0018] Optionally, the first scene perception item is one of a perception item of the target object category, a perception item of the weather condition, and a perception item of the lighting condition.

[0019] Optionally, the at least one scene perception item includes a scene perception item of the target object category, a scene perception item of the target object motion attribute, a scene perception item of the target object position, and a scene perception item of the environmental element; wherein, the scene perception item of the environmental element includes at least one of a scene perception item of the weather condition and a scene perception item of the lighting condition.

[0020] Optionally, determining the set value of each scene perception item in the scene perception item set in the test scenario according to the scene setting information for the test scenario of the intelligent driving function to be tested includes:

[0021] For each scene perception item, match the setting item associated with the scene perception item in the scene setting information;

[0022] Determine the set value of the scene perception item in the test scenario according to the setting item associated with the scene perception item.

[0023] Optionally, injecting the converted perception result data into the intelligent driving controller includes:

[0024] In response to the received injection instruction, inject the converted perception result data into the intelligent driving controller.

[0025] According to a second aspect of the present disclosure, there is provided a test device according to some embodiments, the test device including:

[0026] A processor;

[0027] A memory for storing instructions executable by the processor;

[0028] Wherein, the processor is configured to implement the test method according to the first aspect of the present disclosure when executing the instructions stored in the memory.

[0029] According to a fifth aspect of the present disclosure, there is also provided a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the test method according to the first aspect of the present disclosure is implemented.

[0030] According to the method of the embodiments of the present disclosure, by injecting scene perception result data into the intelligent driving controller of the test vehicle, at least partially replacing the scene perception result data obtained by the test vehicle based on the real scene data collected by its own sensors, in this way, when testing a given test scenario, it is no longer necessary to construct test conditions in the test site according to the scene setting information, reducing the unsafe factors in the test, and significantly improving the test efficiency and reducing the test cost. Moreover, the method of this embodiment can be completed by a general device with data processing capabilities, without the need for expensive professional equipment, which also enables effective control of the test cost. In addition, the method of the embodiments of the present disclosure can provide highly consistent scene perception result data in the same multiple tests, making the test highly repeatable, and thus ensuring the accuracy of the test evaluation.

[0031] Through the following detailed description of the exemplary embodiments of the present invention with reference to the accompanying drawings, other features and advantages of the present disclosure will become clear. Brief Description of the Drawings

[0032] The drawings incorporated in and forming a part of this specification illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0033] Figure 1 is a schematic diagram of a test system to which the method provided by the embodiments of the present disclosure can be applied;

[0034] Figure 2 is a schematic flowchart of a method for testing an intelligent driving system according to some embodiments;

[0035] Figure 3 is a schematic diagram of the composition of a perception result database according to some embodiments;

[0036] Figure 4 is a schematic diagram of the split structure of a test scenario according to some embodiments;

[0037] Figure 5 is a schematic flowchart of a method for testing an intelligent driving system according to some other embodiments;

[0038] Figure 6 is a schematic diagram of the hardware structure of a test device according to some embodiments. Detailed Description of the Embodiments

[0039] Various exemplary embodiments of the present invention will now be described in detail with reference to the drawings. It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present invention.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0041] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods, and devices should be considered as part of the specification.

[0042] In all examples shown and discussed herein, any specific values should be construed as merely exemplary and not as limitations. Thus, other examples of the exemplary embodiments may have different values.

[0043] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, further discussion thereof is not required in subsequent drawings.

[0044] It should be noted that actions such as data collection, storage, use, processing, transmission, provision, disclosure, and deletion involved in this disclosure are carried out on the premise of complying with relevant data protection regulations and policies of the country or region where it is located and with the full authorization of the corresponding data owners.

[0045] This disclosure relates to a technical solution for testing the intelligent driving functions of an intelligent driving system. The intelligent driving system referred to in this disclosure can be an Advanced Driver Assistance Systems (ADAS) or an Autonomous Driving System (ADS), which is not limited herein. The on-road testing of intelligent driving functions can be carried out by a test vehicle VUT equipped with an intelligent driving system driving in a given test scenario. Among them, the test scenario is designed according to the intelligent driving function to be tested, and the test scenario includes the setting of target object elements, and can also include the setting of environmental elements such as weather and lighting.

[0046] Taking the active safety function of the intelligent driving system as an example, in order to ensure the reliability of the active safety function, a large number of active safety tests need to be carried out during the development stage of the active safety algorithm. The active safety tests include, for example, Autonomous Emergency Braking (AEB) tests, Blind Spot Detection (BSD) tests, Autonomous Emergency Steering (AES) tests, Forward Collision Warning (FCW) tests, etc. Among them, according to various standard requirements and its own development needs, each test usually includes multiple test scenarios, and developers need to build a large number of test scenarios in the test site to complete various function tests, resulting in high test costs.

[0047] However, the test costs generated by building test scenarios are only one aspect. Since the intelligent driving systems in the tests are usually not perfect, test equipment such as test vehicles and target objects is likely to be damaged due to collisions, etc. during the tests, which will lead to an increase in test costs and seriously affect test efficiency.

[0048] Furthermore, since the function execution of the intelligent driving system is related not only to the behavior of the target object but also to environmental elements such as weather and lighting, it is difficult to ensure the high consistency of the test scenario when conducting multiple tests based on the same test scenario, thereby affecting the accuracy of test evaluation based on multiple test results.

[0049] To address the safety risk issues brought about by setting up target objects in the test site, some related technologies have proposed applications of simulation testing based on digital twins. The preliminary preparations for this type of simulation testing include: using a scanning device to scan the test site scene, and based on the scanning data, performing scene reconstruction to generate a simulation virtual scene. Then, according to the requirements of the test scenario, target objects and virtual vehicles mapped to the test vehicle are created in the simulation virtual scene to form a virtual test scene. During the test, the test vehicle travels in the test site and sends vehicle state data such as the vehicle position to the simulation software. The simulation software adjusts the state of the virtual vehicle in the virtual test scene based on the vehicle state data of the real vehicle, so that the motion states of the virtual vehicle and the test vehicle are kept consistent. At the same time, the simulation software sends the virtual scene data collected by the virtual vehicle in the virtual test scene to the intelligent driving controller of the test vehicle, enabling the intelligent driving controller to perform scene perception based on the virtual scene data to obtain a scene perception result, and perform vehicle motion control based on the scene perception result, thereby completing the function test for the given test scenario. Although this technology eliminates the work of setting up target objects in the test site, since this technology requires professional personnel to perform scene reconstruction and set up a virtual test scene, and also requires the configuration of high-performance simulation equipment, etc., the test cost still cannot be controlled.

[0050] In view of this, the embodiments of the present disclosure propose a test method for an intelligent driving system. This method can inject scene perception result data adapted to the test scenario into the intelligent driving controller installed in the test vehicle according to the setting of the test scenario, enabling the intelligent driving controller of the test vehicle to complete the function test based on the injected scene perception result data, without fully relying on real scene perception to complete the corresponding function test. Thus, on the one hand, the test efficiency is improved and the control cost is reduced, and on the other hand, the repeatability of the test scenario is improved.

[0051] The test method of the embodiments of the present disclosure can be applied to a test system for performing function tests on an intelligent driving system. Figure 1 Schematically, a test system 100 to which the method provided by the embodiments of the present disclosure can be applied is given. As Figure 1 shown, the test system 100 includes a test vehicle 101 and a test device 102.

[0052] The test vehicle 101 is equipped with the intelligent driving system to be tested. For example, it is equipped with the ADAS to be tested. The intelligent driving system equipped includes Figure 1The intelligent driving controller 1011 in it. The test vehicle 101 can perform driving tasks such as environment perception, decision-making and planning, and control decisions based on the assembled intelligent driving system. The level of intelligent driving can refer to the automotive intelligent grading standard formulated by the Society of Automotive Engineers (SAE). For example, L1 is assisted driving, L2 is partial autonomous driving, L3 is conditional autonomous driving, L4 is highly autonomous driving, and L5 is fully autonomous driving. The above classification methods for the intelligent driving level are only for illustration, and the present disclosure embodiments do not limit the classification criteria and levels of intelligent driving.

[0053] As Figure 1 shown, the test vehicle 101 further includes a sensing component 1014, a communication bus 1012, an electronic control unit (ECU) 1013, and an execution component 1015. The communication bus 1012 can be a CAN bus. The intelligent driving controller 1011 of the test vehicle 101 and each ECU of the whole vehicle are connected to the communication bus 1012 as bus nodes, so that each bus node can transmit data through the communication bus 1012.

[0054] In some examples, the sensing component 1014 can be used to collect information of the movable device itself or the outside. The sensing component 1011 can include a vision sensing unit and a motion sensing unit. The vision sensing unit can include one or more cameras. The motion sensing unit can include a wheel speed meter and / or an Inertial Measurement Unit (IMU). In other examples, the sensing component 1011 can further include a radar, a positioning and navigation unit, etc., which are not limited herein. The radar can include at least one of a lidar, a millimeter wave radar, an ultrasonic radar, or other radars. The wheel speed meter can be of any type, such as an electromagnetic wheel speed meter, an optoelectronic wheel speed meter, a mechanical wheel speed meter, a Hall effect wheel speed meter, or a vision wheel speed meter. The positioning and navigation unit can include at least one of a GPS system, a Beidou system, or other global positioning systems.

[0055] In some examples, the control decision of the intelligent driving controller 1011 can be sent to the vehicle ECU through a message, and the ECU sends a control instruction to the corresponding execution component 1015, so that the execution component 1015 performs the corresponding action, thereby realizing the control of the test vehicle 101.

[0056] In some examples, the execution component 1015 is used to perform corresponding actions based on the control of the ECU, so that the test vehicle 101 completes the motion task. The execution component 1015 can include, for example, a power component, a braking component, a transmission component, a steering component, etc.

[0057] Figure 1The test device 102, the intelligent driving controller 1011, and the ECU in it may each include at least one processor and at least one memory. Each processor may execute alone or jointly the instructions stored in the memory to implement the set functions. The processors in the embodiments of the present disclosure may include at least one of a central processing unit (CPU), a graphic process unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), a micro controller unit (MCU), or other processors. The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read only memory (EEPROM), an erasable programmable read only memory (EPROM), a programmable read only memory (PROM), a read only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0058] The memory of the test device 102 is used to store computer programs or executable instructions. The processor of the test device 102 is configured to implement the test method according to any embodiment of the present disclosure when executing the computer program or the executable instructions.

[0059] It should be noted that Figure 1 the structure of the test vehicle 101 shown in is only schematic. The test vehicle 101 in the embodiments of the present disclosure is not limited to the above structure, and may further include more or fewer components according to needs, or the devices may be combined or split, which is not limited herein.

[0060] Next, in combination with Figure 1 describe each embodiment of the present disclosure.

[0061] <First Embodiment>

[0062] Figure 2 shows a schematic flow chart of a test method for an intelligent driving system according to some embodiments. This method may be implemented by Figure 1 the test device 102 in. As Figure 2 shown, the test method of this embodiment may include the following steps S210 to step S230.

[0063] Step S210: Determine the set value of each scene perception item in at least one scene perception item in the test scenario according to the scene setting information of the test scenario for the intelligent driving function to be tested.

[0064] The test scenario referred to in this embodiment is a scenario defined for the intelligent driving function to be tested, rather than a real existing scenario, nor a physical test scenario arranged in the test site according to the scene setting information.

[0065] In some examples, the intelligent driving function to be tested is, for example, an active safety function, and the corresponding function test is also called an active safety test. Here, since the core goal of the active safety function is to prevent accidents, the intelligent driving system usually triggers the corresponding active safety function, such as the emergency braking function, when there is a collision risk during vehicle driving to avoid accidents. Therefore, most of the test scenarios for testing the active safety function belong to high-risk scenarios. If the active safety function to be tested fails during the test, a collision event is likely to occur. Therefore, using the test method of this embodiment in such test scenarios will be significant in improving test efficiency and reducing test costs.

[0066] Those skilled in the art can understand that the method of this embodiment is not limited to the above-mentioned active safety function, and can also be used for testing other functions, such as the automatic parking function, etc., which are not limited here.

[0067] In this embodiment, at least one scene perception item in step S210 is a scene element related to the intelligent driving function to be tested. That is to say, in application, the intelligent driving controller mainly executes the intelligent driving function to be tested based on the perception results of these scene perception items.

[0068] At least one scene perception item in step S210 constitutes a scene perception item set, and this scene perception item set is adapted to the intelligent driving function to be tested or the test scenario for the intelligent driving function to be tested. In some examples, multiple scene perception item sets can be preset, and each scene perception item set is marked with the test scenario or intelligent driving function it is adapted to. Among them, a scene perception item set can be adapted to one test scenario or multiple test scenarios. In other examples, a general scene perception item set can also be preset, and this scene perception item set can be adapted to different test scenarios. In still other examples, the scene perception item set adapted to the test scenario can also be determined according to the scene setting information of the test scenario, so that the scene perception items in the scene perception item set have corresponding setting items in the scene setting information. Here, at least some of the setting items in the test scenario are used as scene perception items, and the test device 102 generates corresponding sensing result data externally.

[0069] In some examples, the set of scene perception items includes perception items of the target object category attribute, perception items of the target object motion attribute, perception items of the target object position attribute, perception items of weather conditions, perception items of lighting conditions, etc. Among them, the target object category attribute, weather conditions, and lighting conditions belong to static elements that are basically unchanged during the test time, and the target object motion attribute and the target object position attribute are behavioral elements that may change during the test process. The target object motion attribute may include motion speed, motion direction, etc. The target object position is the position of the target object in the body coordinate system of the test vehicle, and the target object position is used to represent the relative position relationship between the target object and the test vehicle. Such a set of scene perception items can basically cover the scene elements required to be tracked by different intelligent driving functions and can meet the requirements of different intelligent driving function tests.

[0070] In this embodiment, the test scene can be set for at least one scene element. For example, the scene setting information of the test scene includes setting information about the target object attribute, and the target object attribute may include the target object category attribute, the target object motion attribute, etc. Among them, the set target object category is, for example, a pedestrian, a cyclist, a passenger vehicle, a truck, an animal, or a cone, etc. For another example, the scene setting information includes setting information about weather conditions, and the weather conditions set in the test scene are, for example, sunny, light rain, heavy rain, foggy, snowy, or dusty, etc. For still another example, the scene setting information includes setting information about lighting conditions, and the lighting conditions set in the test scene are, for example, daytime, nighttime with street lights, nighttime without street lights, nighttime with oncoming vehicles with backlight, or dusk, etc. That is to say, the scene setting information of the test scene directly or indirectly reflects the set values of the test scene for at least one scene element.

[0071] In addition, default values can be set for at least some scene elements. When the test scene adopts the default value for a certain scene element, there is no need to perform an explicit setting in the scene setting information. For example, when the weather conditions or lighting conditions are not explicitly set in the scene setting information, the weather conditions and lighting conditions of the test scene can be determined based on the default settings. The default value of the weather conditions is, for example, sunny, and the default value of the lighting conditions is, for example, daytime, etc., which are not limited herein.

[0072] In this embodiment, the scene setting information of the test scene can be input by the tester through the test device 102 or other devices communicatively connected to the test device 102, or can be determined by the test device 102 based on the scene identifier of the test scene input by the tester through the test device 102 or other devices communicatively connected to the test device 102, or can also be partially input by the tester and partially determined based on the scene identifier input by the tester, etc., which are not limited herein.

[0073] In an example of determining at least part of the scenario setting information based on the scenario identifier, the scenario identifier is mapped to the scenario setting information of the identified test scenario. The test device 102 can parse and / or retrieve at least part of the scenario setting information of the corresponding test scenario based on the scenario identifier, which can reduce the operation burden of the tester.

[0074] In some examples, the scenario setting information of the test scenario includes, for example, the set function to be tested, the set target object category, the set motion attributes of the test vehicle, and the set motion attributes of the target object. The function to be tested is, for example, automatic emergency braking (AEB), automatic emergency steering (AES), or forward collision warning, etc. The set target object category is, for example, pedestrians, two-wheeled vehicles, or vehicles, etc. The set motion attributes of the test vehicle include, for example, the motion speed and motion direction of the test vehicle. The motion direction of the test vehicle is, for example, straight ahead, left turn, or right turn, etc. The set motion attributes of the target object include, for example, the target object motion speed and motion direction. The set motion direction of the target object is, for example, moving longitudinally forward, moving towards the vehicle from the opposite direction ahead, or crossing in front, etc. For a test scenario where the target object is stationary, the set target object motion speed is 0.

[0075] In addition, the scenario setting information of the test scenario can also include the set initial position of the target object or the set action position of the target object. Among them, the action position of the target object is the position of the target object that can trigger the intelligent driving function to be tested according to the settings of the intelligent driving system. Both the initial position of the target object and the action position of the target object can be used to determine the set value of the target object position attribute.

[0076] Taking the test scenario identified by the scenario identifier CPLA - 20kph as an example, this test scenario is a test scenario of a test vehicle moving straight ahead and a pedestrian walking longitudinally in front. The intelligent driving function to be tested is the automatic emergency braking function, the set target object category is pedestrians, the test vehicle is set to move straight ahead at a speed of 20 km / h, and the target object is set to move at a speed of 5 km / h in the same direction as the vehicle's driving direction.

[0077] Taking the test scenario identified by CBLA - 40kph as another example, this test scenario is a test scenario of a test vehicle moving straight ahead and an electric bicycle moving longitudinally in front. The intelligent driving function to be tested is the automatic emergency braking function, the set target object category is electric bicycles, the test vehicle is set to move straight ahead at a speed of 40 km / h, and the target object is set to move at a speed of 15 km / h in the same direction as the vehicle's driving direction.

[0078] In some examples, the test device 102 can provide a test interface for testing the intelligent driving system, and provide at least one setting item in the test interface, enabling the tester to input a scenario identifier and / or scenario setting information through the setting item. The test device 102 executes step S210 based on the received scenario setting information and / or the scenario setting information determined based on the received scenario identifier.

[0079] In this example, the setting item of the test interface can be any form of input box, selection box, etc., which is not limited herein.

[0080] Regarding determining the set value of the scenario perception item in the test scenario, in some examples, the scenario perception item can be matched with the scenario setting information of the test scenario, and based on the set item of the scenario perception item obtained by the matching, the set value of the scenario perception item in the test scenario is determined. This example can determine the set item that matches the scenario perception item based on semantics or a similarity algorithm, which is not limited herein. That is to say, in step S210, determining the set value of each scenario perception item in the test scenario according to the scenario setting information of the test scenario for the intelligent driving function to be tested may include the following steps: for each scenario perception item, matching the set item associated with the scenario perception item in the scenario setting information; and, based on the set item associated with the scenario perception item, determining the set value of the scenario perception item in the test scenario.

[0081] In this example, the scenario setting information can include multiple set items distinguished by different name labels, which can be used to match the scenario perception item with the set item based on the name label, and determine the set value corresponding to the scenario perception item based on the set information of the matched set item, thereby improving the matching efficiency.

[0082] For example, if the set of scene perception items includes perception items of the target object category attribute, and the setting information of the setting item regarding the target object category in the scene setting information indicates that the set target object category is "pedestrian", then the set value of the perception item of the target object category attribute in this test scene is "pedestrian". Another example, if the set of scene perception items includes perception items of weather conditions, and the setting information of the setting item regarding weather conditions in the scene setting information indicates that the set weather condition is "heavy rain", then the set value of the perception item of weather conditions in this test scene is "heavy rain". Another example, if the set of scene perception items includes lighting conditions, and the setting information of the setting item regarding lighting conditions in the scene setting information indicates that the set lighting condition is "street lights at night", then the set value of the perception item of lighting conditions in this test scene is "street lights at night". Another example, if the set of scene perception items includes perception items of the target object motion attribute, and the scene setting information indicates that the set target object motion speed in the corresponding test scene is 5 km / h and the target object motion direction is the same as that of the test vehicle, then the set value of the perception item of the target object motion attribute in this test scene can be expressed as "+5 km / h".

[0083] Regarding the set value of the perception item of the target object position attribute in the test scene, the scene setting information of the test scene may include this set value or may include relevant information that can determine this set value, which is not limited here. For example, the test device 102 can determine the position coordinates of the target object in the current frame based on the setting of the motion direction in the scene setting information as the set value of the perception item of the target object position attribute in this test scene. Another example, when the scene setting information includes the setting of the initial position of the target object, the test device 102 can also determine the position coordinates of the target object in the current frame based on the set initial position of the target object and the set target object motion speed as the set value of the perception item of the target object position attribute in this test scene. Another example, when the scene setting information includes the setting of the action position of the target object, the test device 102 can use this target object action position as the set value of the perception item of the target object position attribute in this test scene. The target object action position can be input by the tester before the test or during the test, which is not limited here.

[0084] Step S220, for each scene perception item, convert the set value of this scene perception item in this test scene into the perception result data corresponding to this test scene for this scene perception item.

[0085] In this embodiment, the perception result data corresponding to each scene perception item conforms to the perception result format requirements of the intelligent driving controller installed in the test vehicle, so that the intelligent driving controller can correctly parse the converted perception result data.

[0086] In this embodiment, the intelligent driving controller configured in the test vehicle VUT is a component used to execute intelligent driving functions in the intelligent driving system. In the related technical field, the intelligent driving controller is also referred to as the intelligent driving domain controller.

[0087] In this embodiment, different scene perception items may have the same perception result format or may have different perception result formats, which are not limited herein.

[0088] The perception result format is, for example, a binary serialization format, a standardized message format, a JSON format, an XML format, etc., which are not limited herein.

[0089] In some examples, after converting the set value of the scene perception item in the test scene into the perception result data corresponding to the test scene for the scene perception item, the first scene perception result data including the perception result data corresponding to each scene perception item can be obtained.

[0090] Regarding obtaining the first scene perception result data for the corresponding test scene based on the perception result data corresponding to each scene perception item, in some examples, according to the encapsulation structure requirements of the intelligent driving controller for the scene perception result, the perception result data corresponding to all scene perception items can be encapsulated or fused into the first scene perception result data. When the intelligent driving system performs scene perception on the scene data collected by the sensor to obtain scene perception data, a unified and structured current frame description will be formed for each frame of scene perception data. In this example, the encapsulation structure requirements of the intelligent driving controller for the scene perception result are also the data structure for the intelligent driving system to perform the current frame description on each frame of scene perception data. In this example, according to such encapsulation structure requirements, all the converted perception result data is encapsulated into the first scene perception result data, and the intelligent driving controller will be able to easily parse and use the externally generated scene perception result data as simply as the intelligent driving controller uses the scene perception result data obtained by performing scene perception based on the sensor data, without the need to use the first scene perception result data through an additional parsing method, improving the adaptability between the intelligent driving controller and the externally generated first scene perception result data and avoiding an additional parsing burden.

[0091] Here, what those skilled in the art can understand is that it is also possible to encapsulate the perception result data corresponding to all scene perception items into the first scene perception result data based on other fusion methods. For example, hierarchically encapsulating the perception result data related to the target object, weather, and light, etc., as long as the intelligent driving controller can correctly parse it, which is not limited herein.

[0092] In some examples, at least one scene perception item includes a first scene perception item, such as a perception item of the target object category attribute, a perception item of the weather condition, or a perception item of the lighting condition, etc., which are static element perception items. In this example, for each scene perception item in step S220, converting the set value of the scene perception item in the test scene into the perception result data corresponding to the test scene for the scene perception item may include: for the first scene perception item, matching, in the perception result database of the first scene perception item, the perception result data corresponding to the set value of the first scene perception item in the test scene as the perception result data corresponding to the test scene for the first scene perception item.

[0093] In this example, the data format of the perception result data in the perception result database conforms to the perception result format requirements of the intelligent driving controller for the first scene perception item. In this way, the perception result data retrieved in the perception result database that matches the set value of the first scene perception item can meet the perception result format requirements of the intelligent driving controller for the first scene perception item.

[0094] In this example, for at least some of the scene perception items, a corresponding perception result database can be established in advance. The perception result database includes the perception result data corresponding to some common set values. That is to say, the perception result database will include multiple perception result data, and different perception result data are adapted to different set values. In this way, when the test device 102 performs the conversion from the set value to the perception result data in step S220, it can complete this conversion by retrieving the perception result data adapted to the set value in the perception result database corresponding to the scene perception item, thereby improving the processing efficiency.

[0095] Since static elements are elements that basically do not change during the test time, have strong stability, and the number of element values is relatively small, and it is also easy to collect from the real scene. Therefore, in this example, this kind of perception result database can be established for the scene perception items regarding static elements, and the static elements are, for example, the target object category, the weather condition, the lighting condition, etc. Correspondingly, the first scene perception item can be one of the perception items of the target object category, the perception item of the weather condition, and the perception item of the lighting condition.

[0096] For behavioral elements such as the target object motion attribute and the target object position attribute, since the number of their values is relatively large and may also be adjusted during the test, etc. Therefore, for behavioral elements, the test device can perform format conversion according to the scene setting information of the current test scene.

[0097] In one example, the perception result data in the perception result database can be extracted from pre-generated second scene perception result data, and the second scene perception result data is obtained by collecting based on real scene data.

[0098] In this example, referring to Figure 3 , real scene data can be pre-collected by a collection device, and any scene perception model can perform scene perception based on the real scene data to obtain corresponding second scene perception result data. Then, the perception result data included in the second scene perception result data is split to extract the perception result data corresponding to different scene perception items, and the extracted perception result data is stored in the perception result database corresponding to the corresponding scene perception item. The collection device here can be a collection vehicle or a vehicle in normal use, which is not limited herein.

[0099] Continuing to refer to Figure 3 , for example, if the second scene perception result data includes perception result data on the category attribute of the target object, perception result data on weather conditions, and perception result data on lighting conditions, then the split perception result data on the category attribute of the target object is stored in the perception result database corresponding to the category attribute of the target object, the split perception result data on weather conditions is stored in the perception result database corresponding to the weather conditions, and the split perception result data on lighting conditions is stored in the perception result database corresponding to the lighting conditions.

[0100] In this example, the perception result data in the perception result database is derived from the real scene data of the real scene. Compared with the method of converting set values into corresponding perception result data based on set algorithms, this kind of real perception result of the real scene is beneficial to improving the accuracy of the test results. In this example, the process of processing the real scene data by the scene perception module to obtain the second scene perception result data can be carried out offline, or at least part of the second scene perception result data that meets the test requirements can be selected from the scene perception result data obtained in the online application, which is not limited herein.

[0101] In this example, by collecting real scene data through the collection device in various real scenes, the perception result databases of each scene perception item can be enriched, so that the perception result data in the perception result database can meet the needs of different test scenes, that is, different test scenes can be adapted by various combinations of different perception result data in each perception result database.

[0102] In order to improve the coverage of the perception result database for different test scenes, when collecting real scene data, the richness of the real scene can be ensured, which can include collecting real scene data in real scenes with non-standard target objects, extreme weather conditions, etc., so as to be able to construct the scene perception result data corresponding to test scenes such as unconventional, high-risk or high test costs through the perception result database, and then be able to complete the functional test of test scenes that are difficult to physically build in a simple way and at a low cost.

[0103] In some examples, the second scene perception result data can be obtained by the intelligent driving system to be tested through scene perception based on real-world data. In this example, the intelligent driving system to be tested is also the intelligent driving system installed in the test vehicle. Through this intelligent driving system, scene perception is performed based on real-world data, and then the second scene perception result data used to construct the perception result database of each scene perception item is obtained. In this way, at least part of the perception result data used to construct the first scene perception result data will be able to reflect the scene perception performance of the intelligent driving system to be tested, which enables the first scene perception result data to be comparable to the expected scene perception result data. Among them, the expected scene perception result is the scene perception data that the intelligent driving system installed in the test vehicle can obtain through scene perception based on the scene data collected by the sensors of the test vehicle when the test vehicle is driving in the physical test scene arranged according to the scene setting information. At this time, the scene perception performance of the scene perception model of the intelligent driving system also becomes a factor affecting the test result as part of the intelligent driving function to be tested, thereby realizing a comprehensive test of the intelligent driving function.

[0104] Step S230: Inject all the converted perception result data into the intelligent driving controller installed in the test vehicle, so that the intelligent driving controller performs motion control of the test vehicle based on the injected perception result data.

[0105] In the example where all the perception result data is encapsulated as the first scene perception result data, this step S230 can inject the first scene perception result data into the intelligent driving controller, so that the intelligent driving controller can perform action responses based on the first scene perception result data. For example, when the first scene perception result data indicates that there is a collision risk between the test vehicle and the target object, the automatic emergency braking function is triggered. Then, according to whether the intelligent driving system can trigger the intelligent driving function to be tested based on the first scene perception result data, the test of the intelligent driving system is completed, and the effect equivalent to testing in the test scene actually arranged according to the scene setting information is obtained.

[0106] In some examples, in step S230, the test device 102 can inject all the perception result data or the first scene perception result data obtained by encapsulation into the intelligent driving controller in response to the received injection instruction. That is to say, in this example, the timing for the test device 102 to inject the first scene perception result data into the intelligent driving controller can be controlled by the tester to improve the operation flexibility and simplify the processing procedure of the test device 102. For example, the tester can input the injection instruction to the test device 102 when the motion condition of the test vehicle meets the scene setting requirements.

[0107] In another example, the test device 102 can also determine the timing of injecting the first scenario perception result data according to a preset test program, which is not limited herein.

[0108] During the test, at least a part of the scenario perception result data used by the intelligent driving controller during the test is externally generated and injected. That is to say, the first scenario perception result data can be all or part of the scenario perception result data required by the intelligent driving controller during the test, which is not limited herein. For example, for scenario elements that are difficult to physically build, they can be externally generated and injected, while for scenario elements that are easy to physically build, such as road elements, etc., they can be determined by the intelligent driving controller through scenario perception based on the sensor data collected by the test vehicle. That is to say, the method of this embodiment can construct the corresponding perception result data externally only for some setting items in the test scenario or some setting items regarding scenario elements, rather than being limited to constructing the corresponding perception result data externally for all setting items or all setting items regarding scenario elements.

[0109] During the test, the intelligent driving controller can collect the state data of the test vehicle through the sensors installed on the test vehicle. The state data of the test vehicle includes the position, movement speed, etc. of the test vehicle, and is used for the movement control of the test vehicle.

[0110] Figure 4 Shows an example of a real vehicle test based on the method of this embodiment. As Figure 4As shown, for a given test scenario, the setting items or scene components of the test scenario can be disassembled based on the scenario setting information to obtain setting values for weather conditions, lighting conditions, target object category attributes, target object motion attributes, target object position attributes, and test vehicle motion attributes. Furthermore, based on the weather condition perception result database, the weather condition setting values are converted into corresponding perception result data; based on the lighting condition perception result database, the lighting condition setting values are converted into corresponding perception result data; based on the target object category attribute perception result database, the target object category attribute setting values are converted into corresponding perception result databases; and based on the set data format requirements, the target object motion attribute setting values and target object position attribute setting values are each converted into corresponding perception result data. The multiple perception result data obtained by conversion are then encapsulated into first scene perception result data, and the encapsulated first scene perception result data is then stored. Finally, based on the set values of the test vehicle's motion properties, the test vehicle's movements can be controlled to meet the corresponding test conditions. Once the test vehicle's actual motion properties reach the set values, the generated first scene perception result data is injected into the intelligent driving controller. In this way, the intelligent driving controller can control the test vehicle's motion based on at least the externally injected first scene perception result data. Testers can then analyze the performance of the tested intelligent driving function based on the motion control results.

[0111] The above steps S210 to S230 illustrate the processing flow for the test device 102 to generate and inject a frame of first scene perception result data into the intelligent driving controller. In a single test, the test device 102 can inject a single frame of first scene perception result data into the intelligent driving controller, or it can inject multiple frames of first scene perception result data in a time-sharing manner, without limitation. In the application of injecting multiple frames of first scene perception result data in a single test in a time-sharing manner, the perception result data of the scene perception item regarding the target object's position attribute in each frame of scene perception result data can change according to the scene settings to achieve target tracking.

[0112] According to the above steps S210 to S230, the method of this embodiment generates and injects the first scene perception result data corresponding to the test scenario into the intelligent driving controller through the test device 102, and cooperates with the actual vehicle to complete the actual vehicle performance test of the intelligent driving function, without the need for the test vehicle to collect scene data through sensors, nor does the intelligent driving controller need to perform scene perception based on the scene data collected by the sensors. Therefore, in this embodiment, the tester does not need to deploy a real test scenario according to the scene setting information, and only a simple and accurate test site is required. For example, when testing the steering function, prepare a test site with intersections, etc., without the need to deploy target objects in the test site according to the scene setting, nor to build the required lighting, weather environment, etc. according to the scene setting. This can not only reduce the cost of building physical test scenarios and actual collision tests, but also reduce the demand for large test sites, lower the cost of site rental and layout, and thus effectively reduce the test cost and improve the test efficiency.

[0113] Moreover, according to steps S210 to S230, the method of this embodiment does not require scene modeling, nor professional simulation equipment, and can be implemented only with a test device having data processing capabilities, which is also a key factor in cost control.

[0114] Furthermore, according to steps S210 to S230, the method of this embodiment can disassemble the scene setting information according to the scene perception item set, so as to combine the perception result data corresponding to the scene setting information of each scene perception item together to form the first scene perception data, which makes the simulation of complex, dangerous, rare or extreme situations, as well as the reproduction of test scenarios simple and easy to implement. Therefore, through the test method of this embodiment, the implementation of the test will no longer be limited by the construction of physical test scenarios, and moreover, when multiple tests need to be performed based on the same test scenario, the high consistency of the test scenario can also be ensured, thereby verifying the performance of the intelligent driving system in various environments and improving the comprehensiveness of the test.

[0115] Furthermore, according to steps S210 to S230, the implementation of the test will not be affected by external environmental factors such as light changes, weather conditions, background interference, etc., ensuring the stability and consistency of the test data. The tester can accurately set and control the values of each perception item through the test device 102, reducing the influence of uncontrollable variables in the test process and improving the reliability of the test results.

[0116] <Second Embodiment>

[0117] In this embodiment, similar to injecting externally generated first scenario perception result data into the intelligent driving controller, the motion attribute data of the test vehicle, etc. can also be generated by the test device 102 based on the scenario setting information and injected into the intelligent driving controller, enabling the intelligent driving control to perform motion control on the test vehicle based on the externally injected first scenario perception result data and the motion attribute data of the test vehicle. Among them, the motion attribute data of the test vehicle includes the motion speed and may also include the motion direction. In this regard, the sensors of the test vehicle for collecting its motion attribute data can be turned off.

[0118] Figure 5 The flowchart of the test method in this embodiment is shown. As Figure 5 shown, the test method may include the following steps S510 to step S530:

[0119] Step S510, according to the scenario setting information of the test scenario for the intelligent driving function to be tested, determine the set value of each scenario perception item in at least one scenario perception item in the test scenario, and the set value of the vehicle state perception item in the test scenario.

[0120] Step S520, for each perception item, convert the set value of the perception item in the test scenario into the perception result data corresponding to the perception item for the test scenario, to obtain the first scenario perception result data including the perception result data corresponding to each scenario perception item, and the ego-vehicle state perception result data including the perception result data corresponding to each state perception item.

[0121] The state perception item is, for example, the motion attribute perception item of the test vehicle.

[0122] Step S530, inject the first scenario perception result data and the ego-vehicle state perception result data into the intelligent driving controller, so that the intelligent driving controller performs the motion control of the test vehicle based on the first scenario perception result data and the ego-vehicle state perception result data.

[0123] In this embodiment, by externally injecting the ego-vehicle state perception result data of the test vehicle, even if there is a deviation between the actual motion speed of the test vehicle during the test and the motion speed required by the scenario setting, the intelligent driving function test for the test scenario can still be completed, which is beneficial to improving the test accuracy.

[0124] <Third Embodiment>

[0125] This embodiment provides a test device, as Figure 6As shown, the test device 600 includes a memory 602 and a processor 601. The memory 602 is used to store the computer program run by the processor 601. The processor 601 is configured to implement the monitoring method according to any embodiment of the present disclosure when executing the computer program stored in the memory 602.

[0126] The test device 600 can be any electronic device with computing and processing capabilities. For example, it can be a laptop computer, a PC, a tablet computer, etc., which is not limited here.

[0127] In addition, an embodiment of the present disclosure also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the monitoring method according to any embodiment of the present disclosure.

[0128] The present invention can be a system, a method, and / or a computer program product. The computer program product can include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0129] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0130] The computer-readable program instructions described herein can be downloaded to various computing / processing devices from a computer-readable storage medium or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0131] The computer program instructions for carrying out operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present invention.

[0132] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0133] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises a manufacture including instructions for implementing various aspects of the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0134] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0135] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, and the module, segment of code, or portion of an instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the boxes may occur out of the order noted in the figures. For example, two consecutive boxes may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box of the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions. As is well known to those skilled in the art, implementation by hardware, implementation by software, and implementation by a combination of software and hardware are equivalent.

[0136] The embodiments of the present invention have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary technical personnel in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A method for testing an intelligent driving system, characterized in that, Including: Determine the set value of each scene perception item in at least one scene perception item in the test scenario according to the scene setting information of the test scenario for the intelligent driving function to be tested; For each scene perception item, convert the set value of the scene perception item in the test scenario into the perception result data corresponding to the test scenario for the scene perception item; wherein, the perception result data corresponding to the scene perception item conforms to the perception result format requirement of the intelligent driving controller assembled on the test vehicle for the scene perception item, and the intelligent driving controller is a component that executes the intelligent driving function in the intelligent driving system; Inject all the converted perception result data into the intelligent driving controller, so that the intelligent driving controller performs motion control of the test vehicle based on the injected perception result data.

2. The method according to claim 1, wherein The injecting all the converted perception result data into the intelligent driving controller includes: According to the encapsulation structure requirement of the intelligent driving controller for the scene perception result, encapsulate the perception result data corresponding to all scene perception items into the first scene perception result data; Inject the first scene perception result data into the intelligent driving controller.

3. The method according to claim 1, characterized in that, The at least one scene perception item includes a first scene perception item, and for each scene perception item, converting the set value of the scene perception item in the test scenario into the perception result data corresponding to the test scenario for the scene perception item includes: For the first scene perception item, match the perception result data corresponding to the set value of the first scene perception item in the test scenario in the perception result database of the first scene perception item as the perception result data corresponding to the test scenario for the first scene perception item; Wherein, the data format of the perception result data in the perception result database conforms to the perception result format requirement of the intelligent driving controller for the first scene perception item.

4. The method according to claim 3, characterized in that The perception result data in the perception result database is extracted from pre-generated second scene perception result data, and the second scene perception result data is obtained through real-scene data collection.

5. The method according to claim 4, wherein The second scene perception result data is obtained by the intelligent driving system through scene perception based on the real-scene data.

6. The method according to claim 3, wherein The first scene perception item is one of the perception items of the target object category, the weather condition, and the lighting condition.

7. The method according to any one of claims 1 to 6, characterized in that The at least one scene perception item includes the scene perception item of the target object category, the scene perception item of the target object motion attribute, the scene perception item of the target object position, and the scene perception item of the environmental element; wherein, the scene perception item of the environmental element includes at least one of the scene perception item of the weather condition and the scene perception item of the lighting condition.

8. The method according to any one of claims 1 to 6, characterized in that The determining the set value of each scene perception item in the scene perception item set in the test scenario according to the scene setting information of the test scenario for the intelligent driving function to be tested includes: For each scene perception item, match the setting item associated with the scene perception item in the scene setting information; Determine the set value of the scene perception item in the test scene according to the set item associated with the scene perception item.

9. The method according to any one of claims 1 to 6, characterized in that The injecting the converted perception result data into the intelligent driving controller includes: In response to the received injection instruction, injecting the converted perception result data into the intelligent driving controller.

10. A testing device, characterized in that, Including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to implement the method according to any one of claims 1 to 9 when executing the instructions stored in the memory.

11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method according to any one of claims 1 to 9 is implemented.

Images