Information exchange method, device, and system

Abstract

The present application discloses an information exchange method, apparatus and system. The method includes the steps of receiving at least one test data, the at least one test data includes gesture information, the gesture information includes a first parameter, the first parameter indicates a first gesture, and the first parameter belongs to a predetermined gesture parameter set, and outputting a first control signal based on the at least one test data, the first control signal is used to control a first virtual terminal. In this way, the ability of the autonomous driving system to identify and respond to traffic police officer gestures can be tested, and the safety and intelligence of autonomous driving can be improved.

Description

[Technical Field]

[0001] The present application relates to the field of intelligent vehicles, and in particular to information exchange methods, devices, and systems. [Background technology]

[0002] In recent years, autonomous driving has become an important development direction in the field of vehicle research and development. To improve the intelligence of autonomous driving, it is necessary to verify the capabilities of the vehicle's autonomous driving system under various operating conditions. In the process of testing an autonomous driving system, various environmental and traffic conditions, such as pedestrians, traffic vehicles, traffic lights, and road infrastructure, must be fully taken into consideration.

[0003] Currently, traffic police officers coordinate and direct traffic in scenarios such as traffic light failure, traffic accident handling, and emergency traffic control. Therefore, identifying and responding to traffic police gestures is important for autonomous vehicle operation. However, currently, there is little research on identifying and responding to traffic police gestures by autonomous driving systems, and there is an urgent need for a solution that can be applied to test the ability of autonomous driving systems to identify and respond to traffic police gestures. Summary of the Invention

[0004] Embodiments of the present application improve the safety and intelligence of autonomous driving by providing information exchange methods, devices, and systems for testing the ability of an autonomous driving system to identify and respond to traffic officer gestures.

[0005] According to a first aspect, there is provided an information exchange method. The method may be applied to a first device. The first device may be any device having an autonomous driving function or an assisted driving function, such as an autonomous driving system (supporting the autonomous driving function or the assisted driving function) or a device or apparatus equipped with the autonomous driving system. The method includes the steps of receiving at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set; and outputting a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal.

[0006] In the aforementioned solution, the autonomous driving system may receive at least one test data and output a first control signal based on the at least one test data to test the autonomous driving system's ability to identify and respond to gestures, thereby improving the safety and intelligence of the autonomous driving of the vehicle.

[0007] In a possible design, the gesture parameter set includes a plurality of parameters, the plurality of parameters indicating any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

[0008] It should be understood that the aforementioned types are merely examples and are not intended to be specific limitations.

[0009] In one possible design, the first parameter is an RGB value, and the RGB value represents an image corresponding to the first gesture. In other words, the test data may be in an image format.

[0010] In this way, the effect of identifying and responding to a first gesture by an autonomous driving system based on imaging techniques can be simulated.

[0011] In a possible design, the first parameter is a true value that indicates a first gesture, for example, the first parameter is a true value agreed upon in the OSI standard.

[0012] In this way, the effect of the autonomous driving system identifying and responding to the first gesture based on the truth value can be simulated.

[0013] In a possible design, the gesture information further includes a second parameter, which indicates information about the lane in which the first virtual terminal is located.

[0014] In this way, information regarding the lane in which the first virtual terminal is located may be provided as part of the gesture information for the autonomous driving system, so that the autonomous driving system can identify and respond to the first gesture by referring to the information regarding the lane in which the first virtual terminal is located, thereby improving the reliability of the solution.

[0015] In a possible design, the gesture information further includes a third parameter, which indicates whether the first parameter is valid.

[0016] In this way, whether to test the autonomous driving system's ability to identify and respond to gestures can be flexibly determined based on requirements, thereby saving computing resources of the autonomous driving system.

[0017] In a possible design, the at least one test data further includes one or more of the following information: target identification information of the first gesture; target position information of the first gesture; the target reference of the first gesture, or Target-driven information for the first gesture.

[0018] In this way, information about the execution target corresponding to the first gesture can be provided to the autonomous driving system, further improving the reliability of the test solution.

[0019] In a possible design, the at least one test data may further include traffic light information.

[0020] In this way, the ability of an autonomous driving system to handle conflicting information (e.g., conflicts between gesture information and traffic light information) may be tested, which may improve the robustness of the autonomous driving system.

[0021] In a possible design, the at least one test data further includes obstacle information.

[0022] In this way, the ability of the autonomous driving system to process interference information is tested, which may improve the robustness of the autonomous driving system.

[0023] In a possible design, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information being indicative of the first gesture.

[0024] In this way, the ability of an autonomous driving system to identify gestures based on V2X communication capabilities can be tested.

[0025] In one possible design, the at least one test data includes gesture information and traffic light information, and correspondingly, outputting the first control signal based on the at least one test data may include outputting the first control signal based on one of the traffic light information and the gesture information, which has a higher priority.

[0026] In this way, when traffic light information and gesture information are received simultaneously, the autonomous driving system may output a first control signal based on the information with higher priority, thereby improving the robustness of the autonomous driving system.

[0027] According to a second aspect, there is provided an information exchange method. The method may be applied to a second device, which may be any device having a simulation function, such as a simulation system. The method includes the steps of outputting at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set; receiving a first control signal from the first device; and controlling a first virtual terminal based on the first control signal.

[0028] In a possible design, the gesture parameter set includes a plurality of parameters, the plurality of parameters indicating any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

[0029] In a possible design, the first parameter is an RGB value, where the RGB value represents an image corresponding to the first gesture, or the first parameter is a true value, where the true value indicates the first gesture.

[0030] In a possible design, the gesture information further includes a second parameter indicating information about the lane in which the first virtual terminal is located, and / or a third parameter indicating whether the first parameter is valid.

[0031] In a possible design, the at least one test data further includes one or more of the following information: target identification information of the first gesture; target position information of the first gesture; the target reference of the first gesture, or Target-driven information for the first gesture.

[0032] In a possible design, the at least one test data further includes at least one of traffic light information or obstacle information.

[0033] In a possible design, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information being indicative of the first gesture.

[0034] For the technical effects of the design in the second aspect, please refer to the technical effects of the corresponding design in the first aspect, and the details will not be described again here.

[0035] In one possible design, when a first event is detected, the virtual object is controlled to perform a first gesture. For example, the first event may include one or more of an event in which the distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, or a traffic accident. It should be understood that the aforementioned types are merely examples and are not limiting.

[0036] In this design, driving the virtual subject to perform gestures based on events is closer to how gestures are triggered in real scenarios, which may improve the reliability of the test.

[0037] In a possible design, the virtual object may be controlled based on the road collection data to perform the same gesture as a gesture in the road collection data, where the road collection data indicates that the real object would perform the first gesture in a real road scenario.

[0038] In this design, driving the virtual object to perform the same gesture as the real gesture is close to the gesture change rules in real scenarios, which may improve the reliability of the test.

[0039] In a possible design, the virtual subject is controlled to perform specified gestures at specified times, the specified gestures including the first gesture.

[0040] In this design, the complexity of the solution can be reduced by driving the virtual object to perform gestures according to pre-set rules.

[0041] According to a third aspect, there is provided an information exchange device comprising a module / unit / technical means configured to perform the method according to the first aspect or any one of the possible designs of the first aspect.

[0042] For example, the apparatus includes a transceiver module and a processing module. The transceiver module is configured to receive at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The processing module is configured to generate a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal. The transceiver module is further configured to output the first control signal.

[0043] According to a fourth aspect, there is provided an information exchange device comprising a module / unit / technical means configured to perform the method according to the second aspect or any one of the possible designs of the second aspect.

[0044] For example, the device includes a transceiver module and a processing module. The transceiver module is configured to output at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The transceiver module is further configured to receive a first control signal from the first device. The processing module is configured to control a first virtual terminal based on the first control signal.

[0045] According to a fifth aspect, there is provided an information exchange device, the information exchange device including a processor and an interface circuit, the interface circuit configured to receive a signal from another device other than the device and send the signal to the processor, or to send a signal from the processor to another communication device other than the device, the processor configured to implement a method according to the first aspect or any one of possible designs of the first aspect by using logic circuitry or executing code instructions.

[0046] According to a sixth aspect, there is provided an information exchange apparatus, the information exchange apparatus including a processor and an interface circuit, the interface circuit configured to receive a signal from another device other than the apparatus and send the signal to the processor, or to send a signal from the processor to another communication device other than the apparatus, the processor configured to implement a method according to the second aspect or any one of possible designs of the second aspect by using logic circuits or executing code instructions.

[0047] According to a seventh aspect, there is provided an information exchange system comprising an apparatus according to the third aspect and an apparatus according to the fourth aspect, or comprising an apparatus according to the fifth aspect and an apparatus according to the sixth aspect.

[0048] According to an eighth aspect, there is provided a computer readable storage medium comprising a program or instructions which, when executed on a computer, enables the performance of a method according to the first aspect, any one of the possible designs of the first aspect, the second aspect, or any one of the possible designs of the second aspect.

[0049] According to a ninth aspect, there is provided a computer program product comprising instructions, the computer program product storing the instructions which, when executed on a computer, enable the computer to perform a method according to the first aspect or any one of the possible designs of the first aspect, or a method according to the second aspect or any one of the possible designs of the second aspect.

[0050] According to a tenth aspect, there is provided a vehicle, the vehicle including the device according to the third aspect, the device according to the fifth aspect, or the computer-readable storage medium according to the eighth aspect. [Brief explanation of the drawings]

[0051] [Figure 1] 1 is a schematic diagram of the architecture of an information exchange system according to an embodiment of the present application; [Figure 2] 1 is a schematic diagram of a possible hypothetical traffic scenario according to an embodiment of the present application; [Figure 3] 1 is a flowchart of an information exchange method according to an embodiment of the present application. [Figure 4] 1 is a schematic diagram of possible test data according to an embodiment of the present application; [Figure 5] 3A-3C are schematic diagrams of virtual object images in the field of view of different virtual terminals according to an embodiment of the present application; [Figure 6] 1 is a schematic diagram of the structure of an information exchange device according to an embodiment of the present application; [Figure 7] FIG. 10 is a schematic diagram of the structure of another information exchange device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0052] Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0053] Embodiments of the present application may be used in any scenario in which the capabilities of a drive system may be tested.

[0054] For example, Figure 1 is a schematic diagram of the architecture of an information exchange system to which embodiments of the present application are applicable. The system includes a second device and a first device. The second device is configured to generate at least one test data and output the at least one test data to the first device. The first device is an object under test (or an object under test is placed on the first device). The first device is configured to receive the at least one test data, process the at least one test data, generate a test result (e.g., a first control signal), and output the test result to the second device.

[0055] In a possible design, the first device may be an autonomous driving system or any device integrated with the autonomous driving system. The specific form of the device may be a chip (e.g., a controller, a processor, or an integrated circuit), a terminal (e.g., an in-vehicle terminal), etc. This is not limited in this application.

[0056] For example, autonomous driving systems in this embodiment of the present application include, but are not limited to, the following systems:

[0057] (1) Emergency assistance system: An emergency assistance system has the ability to continuously detect and respond to several targets and events in a dynamic driving task, rather than continuously performing horizontal or vertical movement control of the vehicle in a dynamic driving task.

[0058] (2) Partial driver assistance system: A partial driver assistance system has the ability to continuously perform horizontal or vertical movement control of a vehicle in a dynamic driving task under operating conditions designed by the partial driver assistance system, and to detect and respond to several targets and events corresponding to horizontal or vertical movement control of the vehicle.

[0059] (3) Combined driver assistance system: The combined driver assistance system has the ability to continuously perform horizontal and vertical movement control of the vehicle in dynamic driving tasks under operating conditions designed by the combined driver assistance system, and to detect and respond to several targets and events corresponding to the horizontal and vertical movement control of the vehicle.

[0060] (4) Conditionally automated driving system: The conditionally automated driving system continuously performs all dynamic driving tasks under the operating conditions designed by the conditionally automated driving system.

[0061] (5) Highly automated driving system: A highly automated driving system continuously performs all dynamic driving tasks and automatically implements the minimum risk policy under the operating conditions designed by the highly automated driving system.

[0062] (6) Fully automated driving system: A fully automated driving system continuously performs all dynamic driving tasks under any driving conditions and automatically implements the minimum risk policy.

[0063] It should be understood that the above types are merely examples and are not specific limitations. The autonomous driving system in the embodiments of the present application is not limited to a country or level. Any system with assisted driving functions or autonomous driving functions falls within the scope of protection of the autonomous driving system in the present application.

[0064] In a possible design, the second device may be a simulation system or any device (such as a chip or terminal) integrated with the simulation system. The second device is configured to generate test data related to traffic scenarios. The test data related to traffic scenarios is used to test the driving system's ability to identify traffic scenarios, its ability to respond to traffic scenarios, etc.

[0065] For a possible design, see Figure 1. The second device includes at least the following three functional modules:

[0066] (1) The scenario establishment module is configured to establish a virtual traffic scenario, for example, to establish environmental components (such as roads or buildings), traffic flows (such as vehicles or pedestrians), traffic lights (such as traffic lights or turn indicators), or virtual objects (such as traffic police officer models). The traffic flows include at least one virtual terminal, for example, a first virtual terminal. Figure 2 is a schematic diagram of a possible virtual traffic scenario according to one embodiment of the present application.

[0067] (2) The data analysis module is configured to obtain the scenario data generated by the scenario establishment module, and analyze the scenario data to obtain test data that meets a preset standard.

[0068] It can be appreciated that test data that meets the pre-set standards can be identified and processed by the first device.

[0069] It can be understood that the preset standard is not particularly limited in this application. For example, the preset standard may be the Open Simulation Interface (OSI) standard established by the Association for Standardization of Automation and Measuring Systems (ASAM). The OSI standard defines a universal interface for connecting the development of autonomous driving functions with various driving simulation frameworks to achieve compatibility. This allows any autonomous driving function to be connected to any simulation tool, and various sensor models can be integrated.

[0070] (3) The vehicle dynamic model is configured to control the first virtual terminal in the virtual traffic scenario based on the control signal transmitted by the first device. For example, the vehicle dynamic model is used to obtain a dynamic response value of the first virtual terminal to the virtual traffic scenario based on the first control signal transmitted by the first device and the current attitude information of the first virtual terminal, and the attitude information of the first virtual terminal is updated based on the vehicle dynamic model, thereby achieving the effect of controlling the autonomous driving of the first virtual terminal by the autonomous driving system.

[0071] For a possible design, see FIG. 1 . The first device includes at least a processing module. The processing module performs calculations on at least one test data and outputs a first control signal used to control a first virtual terminal to simulate the effect of an autonomous driving system identifying an event occurring in a scenario and controlling a vehicle in response to the event. For example, the processing module may execute instructions or data corresponding to an autonomous driving algorithm to perform calculations on the at least one test data and output the first control signal.

[0072] It should be understood that the first device and the second device may be located in the same physical entity or in different physical entities, which is not a limitation in this application.

[0073] It should be understood that the system architecture shown in Figure 1 is merely an example, rather than a specific limitation, and there may be other variations in actual applications.

[0074] 3 is a flowchart of an information exchange method according to an embodiment of the present application. An example is used in which the method is applied to the system shown in FIG. 1. The method includes the following steps:

[0075] S101: A second device generates at least one test data.

[0076] The second device first establishes a virtual traffic scenario. For example, the scenario establishment module in the second device may call data in a scenario database to establish the virtual traffic scenario. The scenario database may provide a large number of typical scenarios to meet test requirements. The data in the scenario database may include static scenario data and dynamic scenario data. The static scenario data is used to describe static components in the traffic scenario, such as environmental components (such as roads). The dynamic scenario data is used to describe dynamic components in the traffic scenario, such as moving traffic flows (such as vehicles or pedestrians), changing traffic lights (such as traffic lights or turn signals), or virtual objects that may perform gestures. The scenario database may be stored in the second device or in a device other than the second device. This is not a limitation in the present application.

[0077] The established virtual traffic scenario may be a three-dimensional virtual traffic scenario model, which includes a three-dimensional model of static components and a three-dimensional model of dynamic components. Figure 2 is a schematic diagram of a virtual traffic scenario, including roads, vehicles, pedestrians, traffic police officers, traffic lights, etc.

[0078] It can be understood that the virtual traffic scenario in this embodiment of the present application includes at least a first virtual terminal and a virtual object. The first virtual terminal is a virtual model of a terminal carrying a test object (e.g., an autonomous driving system), for example, a virtual model of an autonomous driving vehicle. The virtual object can simulate a real person and perform the gestures of a real person. For example, the virtual object can be a virtual traffic police officer model and perform traffic police officer gestures, or the virtual object can be a virtual traffic assistant model and perform traffic assistant gestures. As shown in FIG. 2, an example is used in which the virtual object is a virtual traffic police officer model and the first virtual terminal is a vehicle.

[0079] After the virtual traffic scenario is established, the scenario establishment module may control (or drive) the virtual object to perform a first gesture.

[0080] In a possible design, the scenario establishment module may control (or drive) the virtual object to perform a gesture based on an event, where different events may correspond to the same gesture or different gestures. Events include, but are not limited to, an event in which the distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, a traffic accident, etc. For example, when the distance between the virtual object and the first virtual terminal is less than a preset distance, the object is controlled to perform a deceleration gesture. For example, when a traffic accident occurs in the virtual traffic scenario, the virtual object is controlled to perform a pullover gesture. For example, when a traffic jam occurs in the virtual traffic scenario, the scenario establishment module controls the virtual object to perform a lane change gesture. Correspondingly, when or after a first event occurs, the scenario establishment module controls the virtual object to perform a first gesture. Driving the virtual object to perform a gesture based on an event is closer to how gestures are triggered in a real scenario and may improve the reliability of the test.

[0081] In a possible design, the scenario establishment module may control (or drive) the virtual object to perform the same gesture as the gesture in the road collection data based on the road collection data, where the road collection data indicates a gesture performed by a real object in a real road scenario. For example, the road collection data is a video image, and the duration of the video image is three minutes. The first minute of the video indicates a real traffic police officer performing a stop gesture. The next minute of the video indicates a real traffic police officer performing a left turn gesture. The next minute of the video indicates a real traffic police officer performing a right turn gesture. Correspondingly, the scenario establishment module may control the virtual object to sequentially perform a stop gesture, a left turn gesture, and a right turn gesture within a three-minute duration. Correspondingly, the scenario establishment module controls the virtual object in the road data to perform a first gesture based on a first gesture performed by a real traffic police officer in the road collection data. Driving the virtual object to perform the same gesture as the real gesture is closer to the gesture change rule in a real scenario and may improve the reliability of the test.

[0082] In a possible design, the scenario establishment module may control (or drive) the virtual object to perform a specified gesture at a specified time according to a preset rule. For example, the duration of one test is 10 minutes, and the preset rule is that the 10 gestures shown in Table 1 are performed sequentially, with each gesture having a duration of 1 minute. Correspondingly, the specified gestures include the first gesture. Driving the virtual object to perform gestures according to a preset rule may reduce the complexity of the solution.

[0083] In the process of the virtual object performing the gesture, the data analysis module may obtain scenario data of the virtual traffic scenario, and it can be understood that the scenario data obtained by the data analysis module includes at least relevant data of the virtual object.

[0084] In one possible design, one or more sensor models, including but not limited to an image sensor model or a radar model, are integrated into the data analysis module. The data analysis module can acquire perceptual data of a first virtual terminal on a virtual object based on the sensor model, such as an image or point cloud corresponding to the virtual object. In this way, the scenario data acquired by the data analysis module is closer to the perceptual data of a real vehicle in real life.

[0085] In another possible design, the data analysis module may also directly retrieve data in the scenario database used by the scenario establishment module to establish the virtual traffic scenario and use the data as perception data of the first virtual terminal for the virtual object. In this way, the scenario data more accurately describes the virtual traffic scenario.

[0086] After the data analysis module acquires the scenario data of the virtual traffic scenario, if the scenario data of the virtual traffic scenario is in a preset format, the scenario data may be directly used as at least one test data; or if the scenario data of the virtual traffic scenario is not in a preset format, the data analysis module analyzes the scenario data and converts it into at least one test data in a preset format. Optionally, the preset format is a data format that meets the OSI standard. In this way, a standardized connection between the second device and the autonomous driving system in the first device can be realized.

[0087] In this embodiment of the present application, the at least one test data includes at least gesture information, the gesture information includes a first parameter, the first parameter indicates a first gesture, and the first parameter belongs to a predetermined gesture parameter set. The at least one test data is used to test the ability of the autonomous driving system in the first device to identify and respond to the gesture.

[0088] In one possible design, the first virtual object in the virtual traffic scenario is a virtual traffic police officer model, and the gesture parameter set includes a plurality of parameters, each of the plurality of parameters indicating a corresponding traffic police officer gesture, and different parameters indicating different traffic police officer gestures.

[0089] For example, the plurality of parameters may indicate any of a stop gesture, a go straight gesture, a left turn gesture, a left turn waiting gesture, a right turn gesture, a right turn waiting gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

[0090] It should be understood that the definitions of the gestures of traffic police officers in different countries or regions may be different, so in practical applications, the parameters included in the gesture parameter set may be flexibly designed based on requirements.

[0091] In a possible design, for example, the preset standard is the OSI standard. The first parameter may be a red, green, and blue (RGB) value, where the RGB value represents an image corresponding to the first gesture. For example, the data analysis module acquires a two-dimensional or three-dimensional image of a virtual object in a virtual traffic scenario within the field of view of the first virtual terminal based on the sensor model, and simulates the effect of capturing an image of a traffic police officer by a vehicle in the real world based on the image sensor. Because the OSI standard supports data in an image format, the data analysis module may directly send the acquired image data (RGB values) to the first device as test data.

[0092] In a possible design, for example, the preset standard is the OSI standard. The first parameter may be a true value, and the true value indicates a first gesture. For example, Table 1 is an example of a gesture parameter set. In this example, multiple parameters included in the gesture parameter set are all true values. [Table 1]

[0093] It should be understood that Table 1 is merely an example, not a specific limitation. In practical applications, the types of true values ​​and the gesture types corresponding to each true value can be flexibly set based on requirements.

[0094] Of course, in addition to RGB values ​​and true values, the first parameters may be in another data format specifically determined by the data format supported by the first device.

[0095] S102: The second device outputs at least one test data to the first device, and in response, the first device receives the at least one test data.

[0096] Specifically, the data analysis module transmits at least one test data to the first device.

[0097] S103: The first device generates a first control signal based on at least one test data.

[0098] Specifically, the processing module of the first device identifies an event occurring in the virtual traffic scenario based on at least one test data, and then generates a first control signal based on the event, wherein the first control signal is used to control the first virtual terminal so that the first terminal responds to the event.

[0099] In a possible implementation, the processing module of the first device identifies a gesture (e.g., a first gesture) performed by a virtual object in a virtual traffic scenario based on gesture information (e.g., a first parameter) included in at least one test data, and then generates a first control signal based on the first gesture.

[0100] During a particular implementation, the processing module may integrate one or more of the following functions: perception (e.g., perceiving an event that occurred in a virtual traffic scenario based on test data, e.g., a first gesture), prediction (e.g., predicting an event that will occur in the future in a virtual traffic scenario based on sensing results), planning (e.g., planning a driving trajectory of the first virtual terminal based on sensing results and / or prediction results), control (e.g., generating a control signal used to control the driving trajectory of the first virtual terminal), etc.

[0101] For example, see Figure 2. A first gesture performed by a virtual object at an intersection is a left turn gesture, and a processing module of a first device generates a first control signal used to control a first virtual terminal to drive to turn left.

[0102] S104: The first device outputs a first control signal to the second device, and in response, the second device receives the first control signal.

[0103] S105: The second device controls the first virtual terminal based on the first control signal.

[0104] For example, the event occurring in the virtual traffic scenario is a left turn gesture, and the first control signal is used to control the first virtual terminal to turn left. The vehicle dynamic model obtains a dynamic response value of the first virtual terminal to the virtual traffic scenario (e.g., a driving speed or direction required to execute a left turn of the first virtual terminal) based on the first control signal (left turn signal) sent by the first device and the current attitude information of the first virtual terminal (e.g., the lane in which the first virtual terminal is located, or the distance between the first virtual terminal and the intersection), and updates the attitude information of the first virtual terminal based on the dynamic response value (e.g., controlling the first virtual terminal to drive to turn left).

[0105] It can be understood that after controlling the first virtual terminal, the second device can determine the ability of the autonomous driving system to identify and respond to the first gesture by detecting the status of the first virtual terminal. For example, after the vehicle dynamics model updates the attitude information of the first virtual terminal, the second device determines the updated attitude information of the first virtual terminal to determine whether the autonomous driving system accurately identifies and responds to the first gesture, thereby determining the ability of the autonomous driving system in the first device to identify and respond to the gesture.

[0106] It can be understood that the first gesture is used as an example above. In an actual testing process, the gesture may be constantly changed to test the autonomous driving system multiple times. In the testing process, the algorithm of the autonomous driving system may be further continuously adjusted based on the test results to improve the autonomous driving system's ability to identify and respond to gestures.

[0107] According to S101 to S105, the second device generates test data, and based on the test data, performs a closed-loop test on the ability of the autonomous driving system to identify and respond to traffic police officer gestures, thereby improving the safety and intelligence of the autonomous driving of the vehicle.

[0108] It may be understood that in real life, gestures performed by traffic police officers may be targeted (or directional). For example, gestures performed by traffic police officers generally indicate vehicles driving in a direction facing the traffic police officer. In other words, gestures performed by traffic police officers are valid for vehicles facing the traffic police officer and invalid for vehicles not facing the traffic police officer. Thus, in this embodiment of the present application, the data transmission module may transmit at least one test data to the first device when the first gesture is valid for the first virtual terminal (e.g., a virtual object is facing the first virtual terminal), and / or the processing module of the first device may output a first control signal to the first device when it determines that the first gesture is valid for the first virtual terminal (e.g., a virtual object is facing the first virtual terminal). Of course, in addition to the relative position between the virtual object and the first virtual terminal, the determination condition for determining that the first gesture is valid for the first virtual terminal may further include other determination conditions, which are not limited in the present application.

[0109] In a possible design, the gesture information may further include a second parameter indicating information about a lane in which the first virtual terminal is located. The information about the lane in which the first virtual terminal is located may include, for example, an identifier (assigned_lane_id) of the lane in which the first virtual terminal is located. In other words, the information about the lane in which the first virtual terminal is located may be provided as part of the gesture information for the autonomous driving system, allowing the autonomous driving system to identify and respond to the first gesture by referring to the information about the lane in which the first virtual terminal is located, thereby improving the reliability of the solution.

[0110] In a possible design, the gesture information may further include a third parameter indicating whether the first parameter (is_out_of_service) is valid. Correspondingly, when the third parameter indicates that the first parameter is valid, the first device may generate a first control signal based on the first gesture.

[0111] Optionally, if the third parameter indicates that the first parameter is invalid and the at least one test data further includes other information other than gesture information, the first device may output a control signal based on the other information.

[0112] In this way, whether the gesture information is valid can be set, and whether to test the autonomous driving system's ability to identify and respond to the gesture can be flexibly determined based on requirements, thereby saving computing resources of the first device.

[0113] In a possible design, the at least one test data further includes one or more of the following information:

[0114] (1) Target identification information of the first gesture, i.e., identification information of the target performing the first gesture, for example, identification information of the virtual target shown in FIG. 2 .

[0115] (2) Target position information (police_base) of the first gesture, i.e., the position information of the target performing the first gesture, for example, the position information of the virtual target shown in FIG. 2, where the position information may be longitude and latitude, coordinates, etc., and is not limited in this application.

[0116] (3) a model reference of the first gesture, i.e., source information of the object performing the first gesture, for example, the source information of the virtual object shown in FIG. 2 , where the model reference may optionally be a link used to indicate a file address for storing three-dimensional model data of the virtual object; or

[0117] (4) Source reference of the first gesture, i.e., information used to drive the virtual object to perform the first gesture. For example, when the first gesture is driven by a first event, the source reference of the first gesture may be the first event; when the first gesture is driven by road collection data, the source reference of the first gesture may be the road collection data; or when the first gesture is driven by a preset rule, the source reference of the first gesture may be the preset rule.

[0118] In this way, the second device provides the first device with information related to the execution target corresponding to the first gesture, allowing the first device to identify more information related to the first gesture, thereby further improving the reliability of the test solution.

[0119] In real life, in addition to gesture information, a vehicle may also perceive other information used to indicate traffic rules, such as traffic light information or road sign information. In consideration of this, in a possible design, the at least one test data may further include traffic light information, road sign information, etc. The information may be consistent with the gesture information or may contradict the gesture information. This is not limited in the present application.

[0120] If the traffic light information, road sign information, etc. matches the gesture information (e.g., the gesture information indicates a left turn gesture and the traffic light indicates that traffic may turn left, or the road on which the first virtual terminal is located has a left turn identifier), and the third parameter indicates that the first parameter is valid, the first device may output a first control signal based on any one of the aforementioned information.

[0121] If traffic light information, road sign information, etc. contradicts the first gesture (e.g., the first gesture is a left turn gesture and the traffic light indicates that traffic may proceed straight, or the road on which the first virtual terminal is located has only a straight-through identifier), and the third parameter indicates that the first parameter is valid, the first device may output a first control signal based on the priority information of the aforementioned information.

[0122] For example, if the at least one test data includes gesture information and traffic light information, the first device outputs the first control signal based on the traffic light information or the gesture information, whichever has a higher priority.

[0123] Correspondingly, when the priority of the gesture information is higher than the priority of the traffic light information, the first device generates a first control signal based on the first gesture. For example, when the priority of the traffic light is lower than the priority of the gesture information, the traffic light information indicates going straight, and the gesture information indicates turning left, the first control signal output by the first device is used to control the first virtual terminal to turn left.

[0124] Conversely, when the priority of the gesture information is lower than the priority of the traffic light information, the first device generates a first control signal based on the traffic light information. For example, when the priority of the traffic light is higher than the priority of the gesture information, the traffic light information indicates going straight, and the gesture information indicates turning left, the first control signal output by the first device is used to control the first virtual terminal to go straight.

[0125] Optionally, the priority of different types of information in the at least one test data may be determined according to local traffic rules.

[0126] In this way, the second device can support conflict configuration between gesture information and other information and can test the autonomous driving system's ability to handle conflicting information, thereby improving the robustness of the autonomous driving system.

[0127] In real life, there may be other traffic participants around the vehicle, such as other vehicles, pedestrians, or buildings, which may affect the vehicle's perception of the traffic police officer's gesture. In a possible design, the at least one test data may further include interference information.

[0128] For example, the interference information includes obstacle information. The obstacle information is information about an object (i.e., an obstacle) that blocks the line of sight of the first virtual terminal when the first virtual terminal perceives the virtual object, including, but not limited to, the shape, color, contour, spatial position, etc. of the obstacle. It should be understood that in an occlusion scenario, the gesture information may be partially missing. For example, the test data is an image. See FIG. 4. In the image received by the first device, a part of the virtual object is blocked by another vehicle. Therefore, the gesture information of the virtual object is incomplete.

[0129] Of course, the interference information is not limited to information about obstacles, and may be other information. For example, if the test data is an image, the second device may further simulate the interference caused by the hardware performance of the image sensor in real life by reducing parameters such as the brightness and resolution of the image.

[0130] In this way, the second device can support the configuration of interference information and can test the ability of the autonomous driving system to process the interference information, thereby improving the robustness of the autonomous driving system.

[0131] In one possible design, the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information representing the first gesture, and a relative position between the second virtual terminal and the virtual object that is different from a relative position between the first virtual terminal and the virtual object.

[0132] For example, the test data and the perception data are both images. See FIG. 5. The image of the virtual object in the field of view of the first virtual terminal is different from the image of the virtual object in the field of view of the second virtual terminal. Because the line of sight of the first virtual terminal is blocked by the second virtual terminal, only a portion of the image of the virtual object is present. The line of sight of the first virtual terminal is not blocked by an object. Therefore, a complete image of the virtual object can be obtained.

[0133] Correspondingly, the first device may refer to both an image of the virtual object in the field of view of the first virtual terminal and an image of the virtual object in the field of view of the second virtual terminal to identify a gesture performed by the virtual object.

[0134] Of course, in addition to the sensory data from the second virtual terminal, the at least one test data may further include data from another device, for example, image data collected by a roadside camera, which is not limited in this application.

[0135] In this way, the second device can provide gesture information of a virtual object in a different field of view to the first device, thereby testing the autonomous driving system's ability to identify gestures based on Vehicle to Everything (V2X) communication capabilities.

[0136] It can be understood that the above-mentioned implementations may be implemented separately or in combination with each other, which is not limited in the present application.

[0137] The above describes the method provided in the embodiments of the present application with reference to the accompanying drawings, and the hereinafter describes the apparatus provided in the embodiments of the present application with reference to the accompanying drawings.

[0138] Based on the same technical concept, an embodiment of the present application provides an information exchange device, which includes a module / unit / means configured to execute the method executed by the second device and / or the first device in the above-mentioned method embodiments. The module / unit / means may be implemented by software, hardware, or hardware executing corresponding software.

[0139] See, for example, Figure 6. The device may include a transceiver module 201 and a processing module 202. The device is configured to implement the functionality of a second device or a first device.

[0140] For example, when the device is used as a first device, the transceiver module 201 is configured to receive at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The processing module 202 is configured to generate a first control signal based on the at least one test data, the first control signal being used to control the first virtual terminal. The transceiver module 201 is further configured to output the first control signal.

[0141] Optionally, the gesture parameter set includes a plurality of parameters, the plurality of parameters indicating any of a stop gesture, a go straight gesture, a left turn gesture, a wait left turn gesture, a right turn gesture, a wait right turn gesture, a lane change gesture, a slow down gesture, a pull over gesture, or an unknown gesture.

[0142] Optionally, the first parameter is an RGB value, the RGB value representing an image corresponding to the first gesture, or the first parameter is a true value, the true value indicating the first gesture.

[0143] Optionally, the gesture information further includes a second parameter indicating information about a lane in which the first virtual terminal is located, and / or a third parameter indicating whether the first parameter is valid.

[0144] Optionally, the at least one test data further includes one or more of the following information: target identification information of the first gesture; target position information of the first gesture; the target reference of the first gesture, or Target-driven information for the first gesture.

[0145] Optionally, the at least one test data further includes at least one of the following information: Traffic light information, or Obstacle information.

[0146] Optionally, the at least one test data further comprises sensory data from a second virtual terminal, the sensory data comprising reference gesture information, the reference gesture information being indicative of the first gesture.

[0147] Optionally, the at least one test data includes gesture information and traffic light information. The processing module 202 is configured to generate the first control signal based on priority information of the traffic light information and priority information of the gesture information, which has a higher priority.

[0148] For example, when the device is used as a second device, the transceiver module is configured to output at least one test data to the first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture, the first parameter belonging to a predetermined gesture parameter set. The transceiver module 201 is further configured to receive a first control signal from the first device. The processing module 202 is configured to control a first virtual terminal based on the first control signal.

[0149] Optionally, the gesture parameter set includes a plurality of parameters, the plurality of parameters indicating any of the following gestures: Stop gesture, go straight gesture, left turn gesture, left turn waiting gesture, right turn gesture, right turn waiting gesture, lane change gesture, slow down gesture, pull over gesture, or unknown gesture.

[0150] Optionally, the first parameter is an RGB value, the RGB value representing an image corresponding to the first gesture, or the first parameter is a true value, the true value indicating the first gesture.

[0151] Optionally, the gesture information further includes a second parameter indicating information about a lane in which the first virtual terminal is located, and / or a third parameter indicating whether the first parameter is valid.

[0152] Optionally, the at least one test data further includes one or more of the following information: target identification information of the first gesture; target position information of the first gesture; the target reference of the first gesture, or Target-driven information for the first gesture.

[0153] Optionally, the at least one test data further includes at least one of the following information: Traffic light information, or Obstacle information.

[0154] Optionally, the at least one test data further comprises sensory data from a second virtual terminal, the sensory data comprising reference gesture information, the reference gesture information being indicative of the first gesture.

[0155] Optionally, the processing module 202 is further configured to: control the virtual object to perform a first gesture when a first event is detected, or control the virtual object to perform a gesture that is the same as a gesture in the road collection data based on the road collection data, where the road collection data indicates that the real object will perform the first gesture in a real road scenario, or control the virtual object to perform a specified gesture at a specified time, where the specified gesture includes the first gesture.

[0156] Optionally, the first event includes one or more of an event in which the distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, or a traffic accident.

[0157] It should be understood that all relevant contents of the steps in the foregoing method embodiments can be cited in the functional descriptions of the corresponding functional modules, and the details will not be described again here.

[0158] In a specific implementation, the device may have multiple product forms, some of which are described below.

[0159] See Figure 7. An embodiment of the present application further provides an information exchange device. The device includes at least one processor 301 and an interface circuit 302. The interface circuit 302 is configured to receive a signal from another device other than the device and send the signal to the processor 301, or to send a signal from the processor 301 to another communication device other than the device. The processor 301 is configured to implement a method performed by the second device or the first device by using logic circuits or executing code instructions.

[0160] It should be understood that the processors mentioned in the embodiments of the present application may be implemented using hardware or software. When the processor is implemented using hardware, the processor may be a logic circuit, an integrated circuit, etc. When the processor is implemented using software, the processor may be a general-purpose processor and is implemented by reading software code stored in a memory.

[0161] For example, a processor may be a Central Processing Unit (CPU), another general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0162] It can be understood that the memory referred to in the embodiments of the present application may be volatile memory or nonvolatile memory, or may include volatile memory and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM may be used, such as static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), Synchlink dynamic random access memory (Synchlink DRAM, SLDRAM), and direct Rambus random access memory (Direct Rambus RAM, DR RAM).

[0163] It should be noted that when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, the memory (storage module) may be integrated into the processor.

[0164] It should be noted that memory as described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0165] Based on the same technical concept, an embodiment of the present application further provides a computer-readable storage medium containing a program or instruction, which, when executed on a computer, executes a method executed by a second device or a first device.

[0166] Based on the same technical concept, an embodiment of the present application further provides a computer program product including instructions, which, when executed on a computer, perform a method performed by the second device or the first device.

[0167] Based on the same technical concept, an embodiment of the present application further provides a vehicle including the device shown in FIG. 6 or FIG.

[0168] Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Thus, the present application may use the form of a hardware-only embodiment, a software-only embodiment, or an embodiment having a combination of software and hardware. In addition, the present application may use the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0169] This application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products herein. It should be understood that computer program instructions can be used to implement each process and / or each block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or any other programmable data processing device to create a machine, whereby the instructions executed by the processor of the computer or any other programmable data processing device create an apparatus for implementing the specific functions of one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0170] These computer program instructions may be stored in a computer-readable memory that may instruct a computer or any other programmable data processing device to operate in a particular manner, whereby the instructions stored in the computer-readable memory generate an artifact that includes an instruction apparatus that implements a particular function in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0171] The computer program instructions may alternatively be loaded onto a computer or other programmable data processing device, such that a sequence of operations and steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing one or more specific functions in one or more procedures in the flowcharts and / or one or more blocks in the block diagrams.

[0172] It is obvious that those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. This application is intended to cover these modifications and variations of this application, provided that they fall within the scope of protection defined by the following claims and their equivalent technologies.

Claims

1. 1. A method for exchanging information, comprising: receiving at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object, the first parameter belonging to a predetermined gesture parameter set; outputting a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal; Including, The gesture information includes: a second parameter indicating information about the lane in which the first virtual terminal is located; and / or a third parameter indicating whether the first parameter is valid; further comprising: method.

2. The gesture parameter set includes a plurality of parameters, the plurality of parameters being: Stop gesture, go straight gesture, left turn gesture, left turn waiting gesture, right turn gesture, right turn waiting gesture, lane change gesture, deceleration gesture, pull over gesture to stop on the shoulder, or unknown gesture The method of claim 1 , wherein the gestures are any of the following:

3. the first parameter is an RGB value, the RGB value representing an image corresponding to the first gesture; or the first parameter is a true value, the true value indicating the first gesture.

3. The method according to claim 1 or 2.

4. The at least one test data target identification information, which is identification information of the virtual target that performs the first gesture; target position information, which is position information of the virtual target that performs the first gesture; target reference information indicating source information of the virtual target performing the first gesture; or object driving information used to drive the virtual object to perform the first gesture; The method of claim 1 , further comprising one or more of:

5. The at least one test data Traffic light information, or Obstacle Information The method of claim 1 , further comprising at least one of:

6. 6. The method of claim 1, wherein the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture, and a relative position between the second virtual terminal and the virtual object is different from a relative position between the first virtual terminal and the virtual object.

7. the at least one test data includes the gesture information and the traffic light information; The step of outputting a first control signal based on the at least one test data includes: outputting the first control signal based on information having a higher priority among the traffic light information and the gesture information; The method of claim 5 , comprising:

8. 1. A method for exchanging information, comprising: outputting at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object, the first parameter belonging to a predetermined gesture parameter set; receiving a first control signal from the first device; controlling a first virtual terminal based on the first control signal; Including, The gesture information includes: a second parameter indicating information about the lane in which the first virtual terminal is located; and / or a third parameter indicating whether the first parameter is valid; further comprising: method.

9. The gesture parameter set includes a plurality of parameters, the plurality of parameters being: Stop gesture, go straight gesture, left turn gesture, left turn waiting gesture, right turn gesture, right turn waiting gesture, lane change gesture, deceleration gesture, pull over gesture to stop on the shoulder, or unknown gesture The method of claim 8 , wherein the gestures indicate any of a plurality of gestures from the group consisting of:

10. the first parameter is an RGB value, the RGB value representing an image corresponding to the first gesture; or the first parameter is a true value, the true value indicating the first gesture.

10. The method according to claim 8 or 9.

11. The at least one test data target identification information, which is identification information of the virtual target that performs the first gesture; target position information, which is position information of the virtual target that performs the first gesture; target reference information indicating source information of the virtual target performing the first gesture; or object driving information used to drive the virtual object to perform the first gesture; 11. The method of claim 8, further comprising one or more of:

12. The at least one test data Traffic light information, or Obstacle Information The method of claim 8 , further comprising at least one of:

13. 13. The method of claim 8, wherein the at least one test data further includes sensory data from a second virtual terminal, the sensory data including reference gesture information, the reference gesture information indicating the first gesture, and a relative position between the second virtual terminal and the virtual object is different from a relative position between the first virtual terminal and the virtual object.

14. The method comprises: controlling the virtual object to perform the first gesture when a first event is detected, the first event including one or more of an event in which a distance between the virtual object and the first virtual terminal is less than a preset distance, a traffic jam, or a traffic accident; or controlling the virtual object to perform the same gesture as a gesture in the road collection data based on road collection data, the road collection data indicating that a real object would perform the first gesture in a real road scenario; or controlling the virtual object to perform a specified gesture at a specified time, the specified gesture including the first gesture; 14. The method of any one of claims 8 to 13, further comprising:

15. An information exchange device, a transceiver module configured to receive at least one test data, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object, the first parameter belonging to a predetermined gesture parameter set; a processing module configured to generate a first control signal based on the at least one test data, the first control signal being used to control a first virtual terminal; Equipped with the transceiver module is further configured to output the first control signal; The gesture information includes: a second parameter indicating information about the lane in which the first virtual terminal is located; and / or a third parameter indicating whether the first parameter is valid; further comprising: Device.

16. An information exchange device, a transceiver module configured to output at least one test data to a first device, the at least one test data including gesture information, the gesture information including a first parameter, the first parameter indicating a first gesture to be performed by a virtual object, the first parameter belonging to a predetermined gesture parameter set; wherein the transceiver module is further configured to receive a first control signal from the first device; a processing module configured to control a first virtual terminal based on the first control signal; Equipped with The gesture information includes: a second parameter indicating information about the lane in which the first virtual terminal is located; and / or a third parameter indicating whether the first parameter is valid; further comprising: Device.

17. 8. An information exchange device comprising a processor and an interface circuit, the interface circuit being configured to receive a signal from another device other than the device and transmit the signal to the processor, or to transmit a signal from the processor to another communication device other than the device, the processor being configured to implement the method of any one of claims 1 to 7 by using logic circuits or executing code instructions.

18. 15. An information exchange device comprising a processor and an interface circuit, the interface circuit configured to receive a signal from another device other than the device and transmit the signal to the processor, or to transmit a signal from the processor to another communication device other than the device, the processor configured to implement the method of any one of claims 8 to 14 by using logic circuits or executing code instructions.

19. A computer readable storage medium containing a program or instructions which, when run on a computer, performs the method of any one of claims 1 to 7.

20. A computer readable storage medium containing a program or instructions which, when run on a computer, performs the method of any one of claims 8 to 14.

21. An information exchange system comprising the device according to claim 15 and the device according to claim 16, or the device according to claim 17 and the device according to claim 18.

22. A vehicle comprising the device of claim 15, or the device of claim 17, or the computer-readable storage medium of claim 19.

Images