Information processing method and information processing device
The information processing method adapts intelligent driving systems to diverse traffic environments by using regional and time information to enhance sensing algorithms and neural networks, addressing inefficiencies and improving safety and performance.
Patent Information
- Application Number
- JP2024529554
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-11-19
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2041-11-19
AI Technical Summary
Intelligent driving systems face challenges in adapting to diverse traffic environments due to regional and temporal differences, leading to inefficiencies in algorithm training and deployment, which affect safety and performance.
An information processing method that utilizes regional and time information to enhance the performance of intelligent driving systems by adjusting sensing algorithms and neural network models to adapt to various spatio-temporal scenarios, improving the applicability of sensing systems to wide-ranging environments.
Enhances the performance of intelligent driving systems by enabling them to operate effectively in diverse traffic environments, reducing training and deployment overhead, and improving safety and comfort.
Smart Images

Figure 2024543528000001_ABST
Abstract
Description
[Technical field]
[0001] The present application relates to the field of intelligent driving technology, and in particular to an information processing method and an information processing device. [Background technology]
[0002] With the development of society, more and more machines in modern life are developing towards automation and intelligence. Intelligent vehicles are gradually entering people's daily life. Intelligent driving systems have also become an important part of intelligent vehicles. Intelligent driving systems can be classified into driving assistance systems, automatic driving systems, and the like. In recent years, advanced driving assistant systems (ADAS) play an important role in intelligent vehicles. Advanced driving assistance systems utilize various sensors installed in vehicles to sense the surrounding situation during the driving process of the vehicle, collect data, identify, detect, and track stationary or moving objects, and combine with navigation map data to perform system operation and analysis, thereby enabling the driver to recognize potential dangers in advance, thereby effectively improving the comfort and safety during the driving of the vehicle. ADAS lays the foundation for truly realizing autonomous driving.
[0003] Target functions implemented by autonomous driving include sensing prediction, decision-making and planning, vehicle control, and the like. The sensing system is the "eyes" of the autonomous vehicle and is configured to sense the environment, obstacles, and the like. The sensing system is the basis for whether the autonomous vehicle can drive safely. For example, the sensing system may include various sensors to obtain information about the environment around the vehicle.
[0004] In light of this, how to improve the performance of advanced driver assistance systems or autonomous driving systems has become a technical challenge that needs to be resolved as soon as possible. Summary of the Invention
[0005] The present application provides an information processing method and an information processing device for improving the performance of an intelligent driving system.
[0006] According to a first aspect, the present application provides an information processing method. The method is applied to an information processing device. The information processing device may be, for example, any sensor attached to a terminal. The method includes: acquiring first information, the first information including environmental information of the terminal where the first device is located; receiving second information from a second device, the second information indicating regional information and / or time information of the terminal; and outputting first sensing information for the terminal based on the first information and the second information. According to the method, first, regional information and / or time information of the terminal is received, and then the first sensing information is output based on the environmental information of the terminal where the first device is located and the regional information and / or time information of the terminal, thereby improving the applicability of the sensing system of the terminal to scenarios corresponding to various space-time information. The various space-time information can be used to distinguish scenarios having a wide range of space and / or time. For example, the terminal is a vehicle. According to the method, the performance of the intelligent driving system of the vehicle can be improved.
[0007] In a possible implementation, the step of obtaining the first information includes: detecting environmental information of the terminal; or receiving the first information from the second device.
[0008] In a possible implementation, the region information and / or time information correspond / correspond to a first algorithm, and the first algorithm belongs to a predefined algorithm set. The step of outputting the first sensing information for the terminal based on the first information and the second information includes: outputting the first sensing information for the terminal based on the first algorithm and the first information. In this implementation, a plurality of algorithms corresponding to different region information and / or time information respectively may be predefined. The environmental information of the terminal is processed based on the first algorithm corresponding to the region information and / or time information, and the first sensing information is output. In this way, the applicability of the sensing system of the terminal to scenarios corresponding to various space-time information can be improved, thereby improving the performance of the intelligent driving system.
[0009] In a possible implementation, the regional information and / or the time information are used to construct a first neural network model. The step of outputting the first sensing information for the terminal based on the first information and the second information includes: outputting the first sensing information for the terminal based on the first neural network model. In this implementation, an initial neural network model may be deployed, and the regional information and / or the time information are input to the initial neural network model. In this manner, the initial neural network model may be automatically adjusted to a first neural network model that adapts to scenarios corresponding to various spatial-temporal information (including the regional information and / or the time information), thereby reducing the overhead of training and deployment of the sensing system.
[0010] In a possible implementation, the step of outputting the first sensing information for the terminal based on the first information and the second information includes: sending the first sensing information to a fusion unit, which may fuse the sensing information output by the first device and the sensing information output by another sensor to obtain fusion information with higher reliability.
[0011] In a possible implementation, the region information includes one or more of a continent name, a country name, a region name, a state, or a city, and / or the time information includes time system information, the time system information including one or more of a season, a holiday, a solar term, or a seasonal timekeeping practice.
[0012] In a possible implementation, the first device includes any one of a camera device, a lidar, a pickup device, an ultrasonic sensor, or a millimeter wave radar.
[0013] According to a second aspect, the present application provides an information processing method applied to an information processing device. The information processing device may be, for example, a fusion unit in a terminal. The method includes: receiving at least one portion of first information from at least one first device, the first information including environmental information of the terminal where the first device is located; receiving second information from a second device, the second information indicating regional information and / or time information of the terminal; and outputting first fusion information of the terminal based on at least one portion of the first information and the second information. According to the method, first, regional information and / or time information of the terminal is received, and then, based on the environmental information of the terminal where the first device is located and the regional information and / or time information of the terminal, the first fusion information is output, thereby improving the applicability of the intelligent driving system to scenarios corresponding to various spatial-temporal information. The various spatial-temporal information can be used to distinguish scenarios having a wide range of space and / or time. For example, the terminal is a vehicle. According to the method, the performance of the intelligent driving system of the vehicle can be improved.
[0014] In a possible implementation, the region information and / or time information correspond to at least one first algorithm, and the at least one first algorithm belongs to a predefined algorithm set. The step of outputting the first fusion information for the terminal based on at least one part of the first information and the second information includes: outputting the first fusion information for the terminal based on at least one first algorithm and at least one part of the first information. In this implementation, a plurality of first algorithms corresponding to different region information and / or time information may be predefined. The environmental information of the terminal is processed based on the first algorithm corresponding to the region information and / or time information, and the first fusion information is output. In this way, the applicability of the sensing system of the terminal to scenarios corresponding to various space-time information can be improved, thereby improving the performance of the intelligent driving system.
[0015] In a possible implementation, multiple algorithms in the predefined algorithm set correspond to different sensors and / or correspond to different regional and / or temporal information.
[0016] In a possible implementation, the regional information and / or the temporal information are used to construct a first neural network model. The step of outputting the first fusion information to the terminal based on at least a portion of the first information and the second information includes: outputting the first fusion information to the terminal based on the first neural network model. In this implementation, an initial neural network model may be deployed, and the regional information and / or the temporal information are input to the initial neural network model. In this manner, the initial neural network model can be automatically adjusted to a first neural network model that adapts to scenarios corresponding to various spatial-temporal information, thereby reducing the overhead of training and deployment of the sensing system.
[0017] In a possible implementation, the at least one portion of the first information includes one or more portions of information from at least one of a sensor, a user, a communication module, or a map module.
[0018] In a possible implementation, the region information includes one or more of a continent name, a country name, a region name, a state, or a city, and / or the time information includes time system information, the time system information including one or more of a season, a holiday, a solar term, or a seasonal timekeeping practice.
[0019] In a possible implementation, the first device includes any one of a camera device, a lidar, a pickup device, an ultrasonic sensor, or a millimeter wave radar.
[0020] According to a third aspect, the present application provides an information processing method applied to an information processing device. The information processing device may be, for example, an external module in a terminal or an external module mounted on the terminal. The method includes: determining second information, the second information indicating regional information and / or time information of the terminal; and transmitting the second information to a first device. According to the method, for example, the first device may be any sensor on the terminal, and the regional information and / or time information of the terminal is provided for the sensor. In this manner, the sensor can output the first sensing information based on the environmental information of the terminal where the first device is located and the regional information and / or time information of the terminal. As another example, the first device may be a fusion unit on the terminal, and the regional information and / or time information of the terminal is provided to the fusion unit. In this manner, the fusion unit can output the first fusion information based on the environmental information of the terminal where the first device is located and the regional information and / or time information of the terminal, thereby improving the applicability of the sensing system of the terminal to scenarios corresponding to various spatial-temporal information. The various spatial-temporal information can be used to distinguish scenarios having a wide range of space and / or time. For example, the terminal is a vehicle. According to the method, the performance of the intelligent driving system of the vehicle can be improved.
[0021] In a possible implementation, determining the second information includes: determining the second information based on a user input, the user input being a screen-based input or a voice input; determining the second information based on an operator press; or determining the second information based on information from a map module. In this implementation, the second information can be determined in multiple ways.
[0022] According to a fourth aspect, the present application further provides an information processing device configured to execute the method of the above-mentioned aspect or any one of the possible implementations of the above-mentioned aspect. Specifically, the information processing device may include modules / units configured to execute the method of the above-mentioned aspect or any one of the possible implementations of the above-mentioned aspect. These modules / units may be implemented by hardware or by hardware executing corresponding software.
[0023] According to a fifth aspect, the present application further provides an information processing device, the information processing device including at least one processor and a communication interface. The communication interface is configured to: receive a signal from a communication device other than the information processing device and transmit the signal to the at least one processor; or transmit a signal from the at least one processor to a communication device other than the information processing device. The at least one processor is configured to implement, via a logic circuit or by executing code instructions, the operation steps of the method in the first aspect or any one of the possible implementations of the first aspect, or the operation steps of the method in the second aspect or any one of the possible implementations of the second aspect, or the operation steps of the method in the third aspect or any one of the possible implementations of the third aspect.
[0024] According to a sixth aspect, the present application further provides a terminal, the terminal comprising: an apparatus configured to perform the method of the first aspect or any one of the possible implementations of the first aspect; and an apparatus configured to perform the method of the third aspect or any one of the possible implementations of the third aspect; or the terminal comprising: an apparatus configured to perform the method of the second aspect or any one of the possible implementations of the second aspect; and an apparatus configured to perform the method of the third aspect or any one of the possible implementations of the third aspect.
[0025] According to a seventy-sixth aspect, the present application further provides a computer readable storage medium, the computer readable storage medium storing a computer program which, when executed, performs operation steps of the method of the first aspect or any one of the possible implementations of the first aspect, or performs operation steps of the method of the second aspect or any one of the possible implementations of the second aspect, or performs operation steps of the method of the third aspect or any one of the possible implementations of the third aspect.
[0026] According to a sixth aspect, the present application further provides a computer program product, the computer program product comprising a computer program or instructions which, when executed by a communications device, causes operation steps of the method of the first aspect or any one of the possible implementations of the first aspect to be performed, or causes operation steps of the method of the second aspect or any one of the possible implementations of the second aspect to be performed, or causes operation steps of the method of the third aspect or any one of the possible implementations of the third aspect to be performed. [Brief description of the drawings]
[0027] [Figure 1] FIG. 1 is a schematic diagram illustrating a vehicle detection system according to an embodiment of the present application. [Diagram 2] FIG. 1 is a schematic diagram illustrating a system according to an embodiment of the present application. [Diagram 3] FIG. 1 is a schematic diagram illustrating a system according to an embodiment of the present application. [Figure 4] FIG. 1 is a schematic diagram illustrating a system according to an embodiment of the present application. [Diagram 5] 1 is a schematic flow chart of an information processing method according to an embodiment of the present application. [Figure 6] 1 is a schematic flow chart of an information processing method according to an embodiment of the present application. [Figure 7] 1 is a schematic diagram showing an information processing device according to an embodiment of the present application; [Figure 8]1 is a schematic diagram showing an information processing device according to an embodiment of the present application; [Figure 9] 1 is a schematic diagram showing an information processing device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0028] Hereinafter, the present application will be described in more detail with reference to the accompanying drawings. It is clear that the described embodiments are not all of the embodiments of the present application, but are a part of them. In addition, the specific operation method in the method embodiment may be applied to the device embodiment. In the description of the embodiments in the present application, a person skilled in the art can understand that various numbers such as "first" and "second" in the present application are merely used for distinction to facilitate the description, and are not used to limit the scope of the embodiments of the present application or to represent the order. "Multiple" means two or more. The term "and / or" represents an association relationship between related objects, and indicates that three relationships may exist. For example, A and / or B indicates the following three cases: when only A exists, when both A and B exist, and when only B exists. The character " / " usually indicates an "or" relationship between related objects. "At least one" means one or more. "At least two" means two or more. "At least one", "any one", or similar expressions refer to any combination of these items, including any combination of a singular item (part) or multiple items (parts).
[0029] Currently, an intelligent driving vehicle may sense the environment around the vehicle by using a sensing system. For example, the sensing system may include various sensors and sense information about the environment around the vehicle. However, these sensors collect information about the local environment around the vehicle. Although the traffic environment around the world is almost the same, there are still regional differences in some aspects, especially those closely related to road traffic, such as traffic facilities (such as signs, markings, signals, and signal lights), traffic police (such as clothing, gestures, etc.), and special transport vehicles (such as police cars, ambulances, and fire engines). For autonomous driving sensing systems, regional or time differences in traffic environments pose challenges to the training and deployment of algorithms. Currently, there are two ways to train and deploy these algorithms. In the first method, a large-scale and comprehensive algorithm is trained, and the first method is applicable to various traffic environments around the world. However, this method requires a large amount of annotated data around the world for sample training. In addition, the super-large network places a heavy burden on the on-board computing unit. Therefore, this method is not realistic in terms of the cost of training and deployment. Another method is to train an intelligent driving vehicle that runs in a certain area by using image data of the area. In this method, attention needs to be paid to the driving area of the vehicle during deployment. In addition, when the vehicle goes out of the area that it is familiar with in the training stage, i.e., the application scope of the design, the performance of the algorithm will be degraded, which will affect the safety of intelligent driving. In this method, the intelligent driving vehicle can only run in a certain area, but the traffic scenarios in many areas have various scenario characteristics. For example, the traffic environments in Hong Kong, Zhuhai City in Guangdong Province, Macau, and mainland China regions, as well as continental European countries, have different characteristics. In the second method, the sensing system is difficult to apply to various scenarios with a wide range of space and / or time. Therefore, a method to solve the problem of how to improve the performance of the intelligent driving system is urgently needed.
[0030] From this perspective, an embodiment of the present application provides an information processing method. Any sensor mounted on a terminal outputs first sensing information for the terminal by referring to environmental information of the terminal on which the sensor is placed and regional information and / or time information of the terminal. Alternatively, a fusion unit mounted on the terminal outputs first fusion information for the terminal by referring to environmental information of the terminal on which the fusion unit is placed and regional information and / or time information of the terminal, thereby improving the applicability of the sensing system of the terminal to scenarios corresponding to various space-time information. Various space-time information can be used to distinguish scenarios having a wide range of space and / or time. For example, the terminal is a vehicle, and the method can more easily meet the user's requirements for a vehicle running in scenarios corresponding to various space-time information. Therefore, the method can improve the performance of the intelligent driving system of the vehicle.
[0031] The industry has proposed classification standards for driving automation systems. The driving automation classification standard proposed by the Society of Automotive Engineers International (SAE International) defines six levels, L0 to L5. L0, L1, and L2 driving assistance systems can provide some assistance to the driver. However, regardless of whether the vehicle's driving assistance function is enabled, the driver must always drive the vehicle himself and supervise the assistance functions provided by the driving assistance system, and the driver must perform operations such as steering, braking, or accelerating as necessary to ensure safety. The differences between L0, L1, and L2 assistance functions are as follows: L0 has no driving automation, and L0 assistance functions are limited to warnings and momentary assistance. L1 assistance functions provide steering or braking / acceleration assistance to the driver. L2 assistance functions provide steering and braking / acceleration assistance to the driver. L3 is conditional driving automation. In some cases, the automated driving system can complete some driving tasks and monitor the driving environment. However, the driver is expected to resume driving control at any time. For example, if a function requires driving control, the driver must drive. L4 is highly automated. The automated driving system can complete the driving task in some environments and under certain conditions and monitor the driving environment. L5 is full driving automation. The automated driving system can complete all driving tasks under all conditions.
[0032] In the embodiment of the present application, the intelligent driving system can be classified into ADAS and autonomous driving system. The advanced driver assistance system (ADAS) can be a driving automation system that reaches a level ranging from L0 to L2, and the autonomous driving system is a driving automation system that reaches a level ranging from L3 to L5 or higher.
[0033] It should be noted that the terminal in this embodiment of the present application may be, for example, a vehicle or another device in a vehicle. The other device includes, but is not limited to, an on-board terminal, an on-board controller, an on-board module, an on-board component, an on-board chip, an on-board unit, an on-board radar, or another sensor, such as an on-board camera. The vehicle may use an on-board terminal, an on-board controller, an on-board module, an on-board component, an on-board chip, an on-board unit, an on-board radar, or an on-board camera. Indeed, the terminal may be an intelligent terminal other than a vehicle, or a component disposed in an intelligent terminal other than a vehicle. The intelligent terminal may be an intelligent traffic device, a smart home device, a robot, or the like. For example, the other device includes, but is not limited to, an intelligent terminal, or another sensor, such as a controller, a chip, a radar, or a camera in an intelligent terminal, another component, and the like.
[0034] A terminal may be equipped with multiple sensors. For example, in FIG. 1, the terminal is used as a vehicle for illustration. The vehicle is equipped with a camera device (such as the camera in FIG. 1), a lidar, a millimeter-wave radar (such as the long-range millimeter-wave radar or the medium / short-range millimeter-wave radar in FIG. 1), an ultrasonic sensor, and the like. Thus, by using these sensors, environmental information around the vehicle is obtained. The vehicle analyzes and processes the obtained environmental information to implement functions such as obstacle detection, target recognition, vehicle positioning, route planning, driver monitoring / warning, and the like, to improve the safety, automation, and comfort of vehicle driving. The environmental information obtained by the sensor is mainly objects around the terminal, such as traffic light information, traffic sign information, traffic police, or obstacles. Details will not be described later.
[0035] The camera device is configured to obtain image information of the environment in which the vehicle is located. Currently, multiple cameras may be mounted on the vehicle to obtain information from more angles. LIDAR is an abbreviation for light laser detection and ranging (LiDAR) system, and mainly includes a transmitter, a receiver, and a signal processing unit. The transmitter is a laser transmission mechanism in the LIDAR. After the laser transmitted from the transmitter illuminates the target object, the laser is reflected by the target object, and the reflected light is focused on the receiver through a lens group. The signal processing unit is responsible for controlling the transmission of the transmitter, processing the signal received by the receiver, and calculating information such as the position, speed, distance, and / or size of the target object.
[0036] Millimeter-wave radar can use millimeter waves as a detection medium to measure the distance, angle, and relative speed between the millimeter-wave radar and the object to be measured. Based on the detection range of the millimeter-wave radar, the millimeter-wave radar can be classified into long-range radar (LRR), mid-range radar (MRR), and short-range radar (SRR). LRR is mainly used in scenarios such as adaptive cruise control and braking assistance. LRR does not have high requirements for the width of the detection angle area, i.e., LRR has low requirements for the 3 dB beam width of the antenna. MRR / SRR is mainly used in scenarios such as automatic parking, lane change assistance, and blind spot detection. MRR / SRR has high requirements for the width of the detection angle area, i.e., MRR / SRR has high requirements for the 3 dB beam width of the antenna and requires the antenna to have a low side lobe level. The beam width is used to ensure the detectable angle range. The low side lobes are used to reduce the clutter energy reflected by the ground, reducing the possibility of false alarms and ensuring driving safety. The LRR can be mounted at the front of the vehicle, and the MRR / SRR can be mounted at the four corners of the vehicle. The LRR and MRR / SRR can be used in combination to provide 360-degree coverage around the vehicle.
[0037] The millimeter wave radar may include a housing, and at least one printed circuit board (PCB) is built into the housing. For example, the printed circuit board may include a power PCB and a radar PCB. The power PCB may provide a voltage for use within the radar, and may also provide an interface for communicating with another device and a security function. The radar PCB may perform transmission, reception and processing of millimeter wave signals, and is integrated with components for processing the millimeter wave signals and antennas (transmitting antenna Tx and receiving antenna Rx) for transmitting and receiving millimeter wave signals. The antennas may be formed on the back side of the radar PCB in the form of a microstrip array and are configured to transmit and receive millimeter waves.
[0038] An ultrasonic sensor, also called ultrasonic radar, is a detection device that utilizes ultrasonic detection. The working principle of an ultrasonic sensor is that ultrasonic waves are transmitted to the outside by using an ultrasonic transmitting device, and the ultrasonic waves reflected by an obstacle are received by a receiving device, and the distance is calculated based on the time difference between the reflection and reception of the ultrasonic waves. Currently, the distance calculated by the ultrasonic sensor may be used to prompt the distance between the vehicle body and the obstacle, assist parking, or reduce unnecessary collisions. It should be understood that the above-mentioned sensors are merely examples of sensors that may be configured on the vehicle in this embodiment of the present application, and do not constitute any limitation. In another embodiment, the sensor may include, but is not limited to, the above-mentioned examples.
[0039] In addition to the multiple sensors, a fusion unit and an external module may be further mounted on the above-mentioned terminal. The external module may be a module that can interact with the sensor and / or the fusion unit inside the terminal, such as an electronic control unit, a domain control unit, a multi-domain controller, or a cockpit domain controller. Alternatively, the external module may be another device outside the terminal.
[0040] An embodiment of the present application further provides an information processing system, which includes at least one sensor, a fusion unit, and an external module. In the following system shown in Figure 2 and Figure 3, N sensors are included, and the explanation is given by using an example where N is more than 2. However, the number of sensors is not limited.
[0041] In a possible implementation, in the information processing system shown in FIG. 2, N sensors are connected to an external module by using a first logical interface layer. The first logical interface layer includes a plurality of first logical interfaces, and the external module is connected to each sensor by using various first logical interfaces. For example, each first logical interface may provide one or more of the following information:
[0042] (1) The operational state of the sensor, which is used to define the operational mode of the sensor, for example, an initialization mode, a normal operation mode, or a calibration mode.
[0043] (2) Environmental information, which represents the weather, time, traffic conditions, and the like around the vehicle.
[0044] For example, the environmental information may include the following information: weather conditions, ambient light intensity, temperature, air pressure, relative humidity, road characteristics (e.g., highway or city), and road surface conditions (temperature, waterlogged, frozen, or rough).
[0045] (3) Vehicle state, which is used to represent the dynamic state of the vehicle, which may be understood as the state in which the vehicle is moving. For example, the dynamic state may be described by using kinematic parameters, which include, but are not limited to, speed, acceleration, jerk, attitude, steering direction, and the like.
[0046] (4) An <interface> operation, which is used to define the operation mode of each logical interface and provide additional parameters. The operation mode of a logical interface is, for example, an operation mode (e.g., data is being transmitted) or an idle mode. The additional parameters may be supplementary information of the data transmitted to the sensor by using the logical interface. For example, the transmitted data is image data, and the additional parameters may be the resolution of the image data.
[0047] (5) Local and / or time information.
[0048] As shown in Fig. 2, the N sensors are connected to the fusion unit by using a second logical interface layer, which includes a plurality of second logical interfaces, and the fusion unit is connected to each sensor by using a different second logical interface.
[0049] The second logical interface may provide information based on characteristics and detection level at the object level (e.g., potential moving objects, road objects, stationary objects, etc.), sensor technology specific information, and more supporting information where available.
[0050] In this implementation, the external module may send the vehicle's regional information and / or time information to each of the N sensors by using the first logical interface layer. Each sensor outputs first sensing information to the fusion unit by referring to the environmental information sensed by the sensor and the vehicle's regional information and / or time information. Sensors of different types, different functions, and different mounting positions may obtain redundant, complementary, or contradictory information regarding the environment around the terminal. The fusion unit may perform a fusion process on the first sensing information of the multiple sensors, thereby obtaining a fusion result of the sensing information with high reliability.
[0051] In another possible implementation, in the information processing system shown in FIG. 3, the fusion unit is connected to an external module by using a third logical interface, and the fusion unit is connected to the N sensors by using a second logical interface layer. For a specific implementation of the second logical interface layer in this specification, please refer to the related description of the second logical interface layer in FIG. 2. Details will not be described again in this specification. The external module may send the region information and / or time information of the vehicle to the fusion unit by using the third logical interface. The N sensors may send information about the environment around the vehicle sensed by the N sensors to the fusion unit by using the second logical interface layer. Then, the fusion unit outputs the first fusion information about the vehicle by referring to the region information and / or time information and the environment information of the vehicle.
[0052] In yet another possible implementation, in the information processing system shown in FIG. 4, an external module is connected to the fusion unit by using a third logical interface. The external module is connected to the N sensors by using a first logical interface layer. The fusion unit is connected to the N sensors by using a second logical interface layer. The external module may send the vehicle's regional information and / or time information to the fusion unit by using the third logical interface, and send the regional information and / or time information to the N sensors by using the first logical interface layer.
[0053] The external modules in the above three information processing systems include, but are not limited to, an electronic control unit, a domain control unit, a multi-domain controller, a cockpit domain controller, an in-vehicle terminal, or the like.
[0054] Hereinafter, an information processing method according to an embodiment of the present invention will be described with reference to a flowchart of the method.
[0055] [Embodiment 1] 5 is a schematic flow chart illustrating an information processing method according to an embodiment of the present application. The method is applicable to the system shown in FIG 2. As shown in FIG 5, the method may include the following steps:
[0056] S501: The second device determines second information, and the second information indicates the terminal's regional information (information about the region where the terminal is located) and / or time information (information about the time where the terminal is currently located). In this embodiment of the present application, the regional information and / or time information may also be referred to as spatial-temporal information. Various spatial-temporal information may be used to distinguish scenarios with wide ranges of space and / or time. Details will not be described later.
[0057] Here, the second device may be the external module in Fig. 2. For example, the second device is an electronic control unit, a domain control unit, a multi-domain controller, a cockpit domain controller, or an in-vehicle terminal.
[0058] The region information may include any one or more of the following: continent name, country name, region name, state, or city. A continent name, for example, may be Asia or Europe.
[0059] The time information includes time system information. The time system information includes one or more of a season, a holiday, a solar term, or a seasonal timekeeping practice. The season may be spring, summer, autumn, or winter. The time system information established by each country may be different. For example, the seasonal timekeeping practice may be a solar term, such as the winter solstice, autumn equinox, or the like, as practiced in China. For example, the seasonal timekeeping practice may be daylight saving time or standard time as practiced in the United States. Daylight saving time is practiced from the second Sunday in March to the first Sunday in November each year, and standard time is practiced from the second Monday in November to the third Monday in March of the following year.
[0060] In an embodiment of the present application, in step S501, the second device may determine the second information in multiple implementations.
[0061] In a possible implementation, the second device may determine the second information based on user input, the user input being a screen-based input or a voice input.
[0062] In another possible implementation, the second device may determine the second information based on an operator press.
[0063] In yet another possible implementation, the second device may determine the second information based on information from a map module.
[0064] S502: The second device transmits second information to the first device, and in response, the first device receives the second information from the second device.
[0065] Here, the first device may be any sensor in Fig. 2. For example, the first device may be a camera device, a lidar, a pickup device, an ultrasonic sensor, or a millimeter wave radar. The pickup device may be any device that can capture sound, such as a microphone.
[0066] S503: The first device obtains first information, where the first information includes environmental information of a terminal where the first device is located.
[0067] In this embodiment of the present application, the first device may obtain the first information in a number of implementations.
[0068] In a possible implementation, the first device may detect environmental information of the terminal. For example, the first device may be a millimeter wave radar and detect road information. As another example, the first device may be a camera device and collect image data of the surrounding environment.
[0069] In another possible implementation, the first device may receive the first information from the second device, where the second device may be an external module. For example, the external module may be a large screen and the first device may receive input from the large screen. As another example, the external module may be a microphone and the first device may receive audio input from the microphone.
[0070] S504: The first device outputs first sensing information for the terminal based on the first information and the second information.
[0071] In this embodiment of the present application, in step S504, the first device may output first sensing information to the terminal based on the first information and the second information in multiple possible implementations.
[0072] In a possible implementation, the region information and / or the time information correspond to a first algorithm, the first algorithm belonging to a predefined algorithm set, and the first device outputs first sensing information for the terminal based on the first algorithm and the first information.
[0073] The multiple first algorithms included in the predefined algorithm set can be obtained by training on various regional and / or time information datasets. These datasets have unique scenario features corresponding to the regional and / or time information. The trained algorithms are deployed to the terminal according to requirements. The first algorithms can include, but are not limited to, any one or more of the following: a special vehicle detection algorithm, a special vehicle siren detection algorithm, a police and gesture detection algorithm, a road sign and marking detection algorithm, a traffic light detection algorithm, and the like. When the first algorithm includes multiple algorithms, the first algorithm can be understood as an algorithm package.
[0074] In one example, the second information includes regional information. Specifically, the regional information corresponds to a first algorithm, and the predefined algorithm set may include first algorithms corresponding to multiple parts of the predefined regional information, respectively. The first device may select a first algorithm corresponding to the received regional information from the predefined algorithm set based on the regional information received from the second device, and then process the first information by using the first algorithm to output first sensing information to the terminal. For example, the terminal is a vehicle. The method may improve the applicability of the sensing system of the terminal to various scenarios having a wide spatial range, thereby improving the performance of the intelligent driving system.
[0075] In another example, the second information includes time information. Specifically, the time information corresponds to a first algorithm, and the predefined algorithm set may include first algorithms corresponding to multiple portions of the predefined time information, respectively. The first device may select a first algorithm corresponding to the received time information from the predefined algorithm set based on the time information received from the second device, and then process the first information by using the first algorithm to output first sensing information for the terminal. For example, the terminal is a vehicle. The method may improve the applicability of the sensing system of the terminal to various scenarios having a wide time range, thereby improving the performance of the intelligent driving system.
[0076] In yet another example, the second information includes region information and time information. Specifically, the region information and time information correspond to a first algorithm, and the predefined algorithm set includes first algorithms corresponding to multiple portions of the predefined region information and time information, respectively, and each portion of the predefined region information and the predefined time information corresponds to one first algorithm. The first device may select a first algorithm corresponding to the region information and the received time information from the predefined algorithm set based on the region information and time information received from the second device, and then process the first information by using the first algorithm to output first sensing information for the terminal. For example, the terminal is a vehicle. The method can improve the applicability of the sensing system of the terminal to various scenarios having a wide range of space and / or time, thereby improving the performance of the intelligent driving system.
[0077] In this embodiment, a plurality of algorithms corresponding to different regional information and / or time information may be predefined. The environmental information of the terminal is processed according to a first algorithm corresponding to the regional information and / or time information, and a first sensing information is output. In this way, the applicability of the sensing system of the terminal to scenarios corresponding to different spatial-temporal information can be improved, and the method can improve the performance of the intelligent driving system.
[0078] In another possible implementation, the regional information and / or the time information are used to construct a first neural network model, and the first device outputs first sensing information for the terminal based on the first neural network model and the first information.
[0079] In one example, the second information includes regional information. Specifically, the regional information is used to construct a first neural network model corresponding to the regional information. The first device may input the regional information into an initial model of the neural network, and the initial model is automatically adjusted to a first neural network model that adapts to the regional information. The first information is then used as an input of the first neural network model, and then the first sensing information for the terminal is output. The method can improve the applicability of the sensing system of the terminal to various scenarios having a wide spatial range, thereby improving the performance of the intelligent driving system.
[0080] In another example, the second information includes time information. Specifically, the time information is used to construct a first neural network model corresponding to the time information. The first device may input the time information into an initial model of the neural network, and the initial model is automatically adjusted to the first neural network model that adapts to the time information. After that, the first information is used as an input of the first neural network model, and then the first sensing information for the terminal is output.
[0081] In yet another example, the second information includes regional information and time information. Specifically, the regional information and time information are used to construct a first neural network model corresponding to the regional information and time information. The first device may input the regional information and time information into an initial model of the neural network, and the initial model is automatically adjusted to a first neural network model that adapts to the regional information and time information. The first information is then used as an input of the first neural network model, and the first sensing information for the terminal is then output.
[0082] In this implementation, an initial neural network model can be deployed in a terminal, and regional information and / or time information are input into the initial neural network model. In this way, the initial neural network model can be automatically adjusted to a first neural network model that adapts to scenarios corresponding to different spatial-temporal information, thereby reducing the overhead of training and deployment of the sensing system and improving the performance of the intelligent driving system.
[0083] In this embodiment of the present application, the first device first receives the regional information and / or time information of the terminal, and then outputs the first sensing information based on the environmental information of the terminal where the first device is located and the regional information and / or time information of the terminal, thereby improving the applicability of the sensing system of the terminal to scenarios corresponding to various space-time information. The various space-time information can be used to distinguish scenarios having a wide range of space and / or time. For example, the terminal is a vehicle. According to this method, the performance of the intelligent driving system of the vehicle can be improved.
[0084] In a possible implementation, based on the first two possible implementations, the step of outputting the first sensing information to the terminal in step S504 may include: sending the first sensing information to a fusion unit, whereby the fusion unit performs fusion based on the first sensing information output by the first device and the first sensing information output by another sensor to obtain fusion information.
[0085] In another possible implementation, after step S504, the information processing method may further include steps S505 and S506.
[0086] S505: The first device transmits the first sensory information to the third device, and in response, the third device receives the first sensory information transmitted by the first device.
[0087] Here, the third device is, for example, a fusion unit.
[0088] S506: The third device performs a fusion process on the first sensing information output by the first device and the sensing information output by another sensor to obtain fusion information.
[0089] In some other embodiments, reference may be made to the information processing system shown in Fig. 4. The difference between the implementation of this embodiment and the implementation of the embodiment shown in Fig. 5 is that after step S504, the fusion unit may alternately receive regional information and / or time information from the second device, and the fusion unit may further perform fusion processing on the first sensing information transmitted by each sensor based on the received regional information and / or time information.
[0090] In this embodiment of the present implementation, the fusion unit may fuse the sensory information output by the first device with the sensory information output by another sensor to obtain fusion information with higher reliability.
[0091] [Embodiment 2] 6 is a schematic flow chart illustrating an information processing method according to an embodiment of the present application. The method is applicable to the system shown in FIG 3. As shown in FIG 6, the method may include the following steps:
[0092] S601: The second device determines second information, and the second information indicates the terminal's regional information (information about the region where the terminal is located) and / or time information (information about the time where the terminal is currently located). In this embodiment of the present application, the regional information and / or time information may also be referred to as spatial-temporal information. Various spatial-temporal information may be used to distinguish scenarios with wide ranges of space and / or time. Details will not be described later.
[0093] Here, the second device may be the external module in Fig. 3. For example, the second device is an electronic control unit, a domain control unit, a multi-domain controller, a cockpit domain controller, or an in-vehicle terminal.
[0094] For specific implementation of the region information or time information, please refer to the relevant description of the region information or time information in step S401, and the details will not be described again in this specification.
[0095] In an embodiment of the present application, in step S601, the second device may determine the second information in multiple implementations.
[0096] In a possible implementation, the second device may determine the second information based on user input, the user input being a screen-based input or a voice input.
[0097] In another possible implementation, the second device may determine the second information based on an operator press.
[0098] In yet another possible implementation, the second device may determine the second information based on information from a map module.
[0099] S602: The second device transmits the second information to the third device, and in response, the third device receives the second information from the second device.
[0100] Here, the third device may be the fusion unit in FIG.
[0101] S603: The third device receives at least one portion of first information from at least one first device, where the first information includes environmental information of a terminal in which the first device is located.
[0102] Here, the first device may be any sensor in FIG. 3. For example, the first device may be a camera device, a lidar, a pickup device, an ultrasonic sensor, a millimeter wave radar, an inertial navigation system, or a GNSS. The pickup device may be any device that can capture sound, such as a microphone. The first device may alternatively be a large screen, a communication module, a map module, or the like. In a possible implementation, the at least one portion of the first information in step S603 includes one or more portions of information from at least one sensor, a user, a communication module (such as a V2X module that may directly or indirectly exchange information with a third device), or a map module.
[0103] S604: The third device outputs first fusion information to the terminal based on at least a portion of the first information and the second information.
[0104] In this embodiment of the present application, in step S604, the third device outputs first fusion information to the terminal based on at least a portion of the first information and the second information in multiple possible implementations.
[0105] In a possible implementation, the region information and / or time information correspond to at least one first algorithm, and the at least one first algorithm belongs to a predefined algorithm set. The third device may output first fusion information for the terminal based on the at least one first algorithm and at least one part of the first information. In this manner, a plurality of first algorithms corresponding to various region information and / or time information may be predefined. The environmental information of the terminal is processed based on the first algorithm corresponding to the region information and / or time information, and the first fusion information is output, thereby improving the applicability of the sensing system of the terminal to scenarios corresponding to various space-time information. The various space-time information may be used to distinguish scenarios having a wide range of space and / or time, thereby improving the performance of the intelligent driving system.
[0106] In this embodiment of the present application, the multiple algorithms included in the predefined algorithm set correspond to different sensors and / or the multiple algorithms correspond to different regional and / or time information. Specifically, the multiple algorithms included in the predefined algorithm set correspond to different sensors, or the multiple algorithms included in the predefined algorithm set correspond to different regional and / or time information, or the multiple algorithms included in the predefined algorithm set correspond to different sensors and different regional and / or time information.
[0107] The multiple first algorithms included in the predefined algorithm set can be obtained by training on various regional and / or time information datasets. These datasets have unique scenario features corresponding to the regional and / or time information. The trained algorithms are deployed to the terminal according to requirements. The first algorithms can include, but are not limited to, any one or more of the following: a special vehicle detection algorithm, a special vehicle siren detection algorithm, a police and gesture detection algorithm, a road sign and marking detection algorithm, a traffic light detection algorithm, and the like. When the first algorithm includes multiple algorithms, the first algorithm may also be understood as an algorithm package.
[0108] In one example, the second information includes regional information, and the regional information corresponds to one first algorithm. Specifically, the first algorithm may process the first information from various sensors. The third device may select a first algorithm corresponding to the received regional information from a predefined algorithm set based on the regional information received from the second device, and then process at least a portion of the first information from each sensor by using the first algorithm to output first fusion information for the terminal. Similarly, the second information includes time information, and the time information corresponds to the first algorithm. For specific implementation, please refer to this example. The second information includes regional information and time information, and the regional information and the time information correspond to one first algorithm. For specific implementation, please refer to this example. Details will not be described again in this specification.
[0109] In another example, the second information includes regional information, and the regional information corresponds to a plurality of first algorithms. The plurality of first algorithms process the first information from different sensors, respectively. For example, the first algorithm 1 is used to process the first information from the camera device, the first algorithm 2 is used to process the first information from the lidar, and the first algorithm 3 is used to process the first information from the millimeter wave radar. In this example, the third device may select a plurality of first algorithms corresponding to the received regional information from a predefined algorithm set based on the regional information received from the second device, and then process at least a portion of the first information from each sensor by using the plurality of first algorithms to output the first fusion information for the terminal. Similarly, the second information includes time information, and the time information corresponds to a plurality of first algorithms. For specific implementation, please refer to this example. The second information includes regional information and time information, and the regional information and the time information correspond to a plurality of first algorithms. For specific implementation, please refer to this example. Details will not be described again in this specification.
[0110] In another possible implementation, the regional information and / or the time information are used to construct a first neural network model. The third device outputs first fusion information to the terminal based on the first neural network model and the first information. In this way, an initial neural network model is deployed, and the regional information and / or the time information are input into the initial neural network model, so that the initial neural network model can automatically adjust to a first neural network model that adapts to scenarios corresponding to various spatial-temporal information, thereby improving the performance of the intelligent driving system and reducing the overhead of training and deployment of the sensing system.
[0111] In one example, the second information includes regional information. Specifically, the regional information is used to build one first neural network model corresponding to the regional information. The first device may input the regional information into an initial model of the neural network, and the initial model is automatically adjusted to a first neural network model that adapts to the regional information. Thereafter, the first information is used as an input of the first neural network model, and then the first fusion information for the terminal is output. Similarly, the second information includes time information, and the time information is used to build one first neural network model corresponding to the regional information. For specific implementation, please refer to the example in which the second information includes regional information. The second information includes regional information and time information, and the regional information and time information are used to build one first neural network model corresponding to the regional information and the time information. For specific implementation, please refer to the example in which the second information includes regional information. Details will not be described again in this specification.
[0112] In another example, the second information includes regional information. Specifically, the regional information is used to construct a plurality of first neural network models corresponding to the regional information, and different first neural network models correspond to different sensors. The first device may input regional information to a plurality of initial models of the neural network corresponding to different sensors. For example, a camera device, a lidar, and a millimeter wave radar are mounted on the terminal. The initial model corresponding to the camera device is input with regional information. The initial model corresponding to the camera device is automatically adjusted to a first neural network model that adapts to the regional information and the camera device. The initial model corresponding to the lidar is input with regional information. The initial model corresponding to the lidar is automatically adjusted to the first neural network model that adapts to the regional information and the lidar. The initial model corresponding to the millimeter wave radar is input with regional information. The initial model corresponding to the millimeter wave radar is automatically adjusted to the first neural network model that adapts to the regional information and the millimeter wave radar. Then, environmental information from the camera device is input to the first neural network model that adapts to the regional information and the camera device, and a first processing result corresponding to the regional information and the camera device is output. The environmental information from the lidar is input to the first neural network model adapted to the regional information and the lidar, and a second processing result corresponding to the regional information and the millimeter wave radar is output. The environmental information from the millimeter wave radar is input to the first neural network model adapted to the regional information and the millimeter wave radar, and a third processing result corresponding to the regional information and the millimeter wave radar is output. Then, the fusion unit performs fusion on the first processing result, the second processing result, and the third processing result, and outputs the first fusion information to the terminal. Similarly, the second information includes time information, and the time information is used to build a plurality of first neural network models corresponding to the regional information. For a specific implementation, please refer to an example in which the second information includes regional information. The second information includes regional information and time information, and the regional information and time information are used to build a plurality of first neural network models corresponding to the regional information and the time information.For a specific implementation, please refer to the example in which the second information includes region information, and the details will not be described again in this specification.
[0113] In some other embodiments, reference may be made to the information processing system shown in Fig. 4. The difference between the implementation of this embodiment and the embodiment shown in Fig. 6 is that before step S603, a first device (e.g., a sensor, etc.) alternately receives regional information and / or time information from a second device, and then the first device processes the first information based on the regional information and / or time information, and then transmits the processed first information to a third device (e.g., a fusion unit, etc.).
[0114] In this embodiment of the present application, a third device (such as a fusion unit) first receives regional information and / or time information of the terminal, and then refers to the environmental information and the regional information and / or time information of the terminal to output first fusion information, which can improve the applicability of the intelligent driving system to scenarios corresponding to various space-time information, and thereby improve the performance of the intelligent driving system.
[0115] Based on the above-mentioned embodiment and similar concepts, the embodiment of the present application further provides an information processing device, configured to execute the steps executed by the first device in the above-mentioned method embodiment. For related features, please refer to the above-mentioned method embodiment. Details will not be described again in this specification.
[0116] As shown in FIG. 7, an information processing device 700 may include: an acquisition unit 701, a communication unit 702, and a processing unit 703.
[0117] The acquiring unit 701 is configured to acquire first information, the first information including environmental information of a terminal where the first device is located.
[0118] The communication unit 702 is configured to receive second information from a second device, the second information indicating local information and / or time information of the terminal.
[0119] The processing unit 703 is configured to output first sensing information for the terminal based on the first information and the second information.
[0120] In a possible implementation, the obtaining unit 701 is specifically configured to: detect environmental information of the terminal; or receive first information from a second device.
[0121] In a possible implementation, the region information and / or the time information correspond to a first algorithm, and the first algorithm belongs to a predefined algorithm set. The processing unit 703 is specifically configured to output first sensing information for the terminal based on the first algorithm and the first information.
[0122] In a possible implementation, the region information and / or the time information is used to construct a first neural network model. The processing unit 703 is specifically configured to output first sensing information for the terminal based on the first neural network model.
[0123] In a possible implementation, the communication unit 702 is specifically configured to transmit the first sensing information to the fusion unit.
[0124] The embodiment of the present application further provides an information processing device, configured to execute the steps executed by the second device or the third device in the above-mentioned method embodiment. For related features, please refer to the above-mentioned method embodiment. Details will not be described again in this specification. As shown in FIG. 8, the information processing device 800 may include a communication unit 801 and a processing unit 802.
[0125] When the information processing device is configured to execute the steps performed by the third device in the above-mentioned method embodiments, the communication unit 801 is configured to: receive at least one portion of first information from at least one first device, the first information including environmental information of the terminal where the first device is located; and receive second information from a second device, the second information indicating regional information and / or time information of the terminal. The processing unit 802 is configured to output first fusion information for the terminal based on the at least one portion of the first information and the second information.
[0126] In a possible implementation, the region information and / or the time information corresponds to at least one first algorithm, and the at least one first algorithm belongs to a predefined algorithm set. Outputting first fusion information to the terminal based on the at least one first algorithm and the at least one portion of the first information.
[0127] In a possible implementation, multiple algorithms in the predefined algorithm set each correspond to a different sensor and / or each correspond to different regional and / or temporal information.
[0128] In a possible implementation, the regional information and / or the temporal information is used to construct a first neural network model. The processing unit 802 is specifically configured to: Outputting first fusion information to the terminal based on the first neural network model.
[0129] In possible implementations, the at least one portion of the first information includes one or more portions of information from at least one of a sensor, a user, a communication module, or a map module.
[0130] In a possible implementation, the region information includes any one or more of the following: a continent name, a country name, a region name, a state, or a city, and / or the time information includes time system information, the time system information including one or more of a season, a holiday, a solar term, or a seasonal timekeeping practice.
[0131] In a possible implementation, the first device includes any one of the following: a camera device, a lidar, a pickup device, an ultrasonic sensor, or a millimeter wave radar.
[0132] When the information processing device is configured to execute the steps performed by the second device in the above-mentioned method embodiments, the processing unit 802 is configured to determine second information. The second information indicates the region information and / or time information of the terminal. The communication unit 801 is configured to transmit the second information to the first device or the third device.
[0133] In a possible implementation, the processing unit 802 is specifically configured to: determine the second information based on a user input, the user input being a screen-based input or a voice input; determine the second information based on an operator press; or determine the second information based on information from a map module.
[0134] According to the above-mentioned method, FIG. 9 is a schematic diagram showing the structure of an information processing device according to one embodiment of the present application invention. As shown in FIG. 9, the information processing device 900 can be the first device, the second device, or the third device in the above-mentioned embodiment. The information processing device 900 can include a memory 901 and a processor 902, and can further include a bus system. The processor 902 and the memory 901 can be connected by using the bus system.
[0135] It should be understood that the processor 902 may be a chip. For example, the processor 902 may be a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processing circuit (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or another integrated chip.
[0136] In the implementation process, the steps of the above-mentioned method can be achieved by using hardware integrated logic circuits in the processor 902 or by using instructions in the form of software. The steps of the method disclosed with reference to the embodiments of the present application may be directly executed by a hardware processor, or may be executed by using a combination of hardware and software modules in the processor 902. The software modules may be arranged in a storage medium well developed in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is arranged in the memory 901. The processor 902 reads the information in the memory 901 and completes the steps of the above-mentioned method in combination with the hardware of the processor 902.
[0137] It should be noted that the processor 902 in this embodiment of the present application may be an integrated circuit chip and has signal processing capabilities. In the implementation process, the steps in the above-mentioned method embodiments are implemented by using hardware integrated logic circuits in the processor or by using instructions in the form of software. The processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or performed. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the methods disclosed with reference to the embodiments of the present application may be directly performed by a hardware decoding processor, or may be performed by a combination of hardware modules and software modules in the decoding processor. The software modules may be located in a storage medium well developed in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is placed in a memory, and the processor reads the information in the memory and completes the steps of the above-mentioned method in combination with the processor's hardware.
[0138] It can be understood that the memory 901 in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include a volatile memory and a non-volatile memory. The non-volatile memory may be a read-only memory (ROMM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which functions as an external cache. By way of example and non-limiting explanation, RAM in many forms is available, such as: Namely, static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchlink dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus random access memory (direct rambus RAM, DR RAM). It should be noted that the memory of the systems and methods described herein includes, but is not limited to, these and any other suitable types of memory.
[0139] The present application further provides a vehicle, which may include the above-mentioned information processing device. In one example, the vehicle may be the first vehicle in the present application.
[0140] According to the method provided in the embodiments of the present application, the present application further provides a computer program product, which includes computer program codes or instructions, which, when executed on a computer, enable the computer to perform the method in any one of the embodiments.
[0141] According to the method provided in the embodiment of the present application, the present application further provides a computer-readable storage medium, which stores a program code, which, when executed on a computer, enables the computer to execute the method in any one of the above-mentioned method embodiments.
[0142] All or part of the above-described embodiments may be implemented using software, hardware, firmware, or any combination thereof. When software is used to implement the embodiments, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the procedures or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired (e.g., coaxial cable, optical fiber, or digital subscriber line (DSL)) manner or in a wireless (e.g., infrared, radio, or microwave) manner. The computer-readable storage medium may be any available medium that can be accessed by a computer, or may be a data storage device, for example, a server or a data center in which one or more available media are integrated. The available media may be magnetic media (such as a floppy disk, hard disk, or magnetic tape), optical media (such as a DVD), or semiconductor media, which may be a solid state drive (SSD).
[0143] Those skilled in the art can clearly understand that for the sake of simple and concise description, the detailed work processes of the above-mentioned systems, devices, and units may be referred to the corresponding processes in the above-mentioned method embodiments, and the details will not be described again in this specification.
[0144] In some embodiments provided in the present application, it should be understood that the disclosed system, device, and method may be implemented in other manners. For example, the described device embodiment is merely exemplary. For example, the division into units is merely a division of logical functions, and may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some functions may be ignored or not performed. Furthermore, the shown or described mutual couplings, or direct couplings, or communication connections may be indirect couplings or communication connections by using some interfaces, devices, or units, and may be connections in electrical, mechanical, or other forms.
[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to the actual requirements to achieve the objectives of the technical solutions of the present application.
[0146] The above description is merely a specific implementation of the present application. Any variations or replacements that are easily conceived by those skilled in the art based on the specific implementations provided in the present application shall fall within the protection scope of the present application.
Claims
1. 1. An information processing method, comprising: A step of acquiring first information, the first information including environmental information of a terminal on which the first device is located; receiving second information from a second device, the second information indicating regional information and / or time information of the terminal; outputting first sensing information for the terminal based on the first information and the second information; A method comprising:
2. The step of acquiring the first information includes: detecting the environmental information of the terminal; or receiving the first information from the second device; The method of claim 1 , comprising:
3. the region information and / or the time information corresponds to a first algorithm, the first algorithm belonging to a predefined set of algorithms; The step of outputting the first sensing information for the terminal based on the first information and the second information includes: outputting the first sensing information for the terminal based on the first algorithm and the first information; Including, The method according to claim 1 or 2.
4. the regional information and / or the temporal information is used to construct a first neural network model; The step of outputting the first sensing information for the terminal based on the first information and the second information includes: outputting the first sensing information for the terminal based on the first neural network model; Including, 4. The method according to any one of claims 1 to 3.
5. The step of outputting the first sensing information for the terminal based on the first information and the second information includes: transmitting the first sensing information to a fusion unit; 5. The method of claim 1, further comprising:
6. 1. An information processing method, comprising: receiving at least one portion of first information from at least one first device, the first information including environmental information of a terminal in which the first device is located; receiving second information from a second device, the second information indicating regional information and / or time information of the terminal; outputting first fusion information of the terminal based on at least a portion of the first information and the second information; A method comprising:
7. the region information and / or the time information corresponds to at least one first algorithm, the at least one first algorithm belonging to a predefined set of algorithms; The step of outputting first fusion information for the terminal based on at least a portion of the first information and the second information includes: outputting the first fusion information for the terminal based on the at least one first algorithm and at least one portion of the first information; Including, The method according to claim 6.
8. The method of claim 7 , wherein multiple algorithms in the predefined set of algorithms correspond to different sensors and / or correspond to different regions and / or times.
9. the regional information and / or the temporal information is used to construct a first neural network model; The step of outputting first fusion information for the terminal based on at least a portion of the first information and the second information includes: outputting the first fusion information to the terminal based on the first neural network model; Including, 9. A method according to any one of claims 6 to 8.
10. 10. The method of claim 6, wherein the at least one portion of the first information includes one or more of information from at least one of a sensor, a user, a communication module, or a map module.
11. The locality information includes one or more of a continent, a country, a region, a state, or a city; and / or the time information includes time system information, the time system information including one or more of a season, a holiday, a solar term, or a seasonal timekeeping practice; 10. The method according to any one of claims 1 to 9.
12. The first device comprises: Camera device, LiDAR, pickup device, ultrasonic sensor, or millimeter wave radar 11. The method of claim 1, further comprising:
13. 1. An information processing method, comprising: determining second information, the second information indicating regional information and / or time information of the terminal; transmitting the second information to a first device; A method comprising:
14. The step of determining the second information further comprises: determining the second information based on a user input, the user input being a screen-based input or a voice input; or determining the second information based on an operator press; or determining said second information based on information from a map module; The method of claim 13, comprising:
15. An information processing device, An acquisition unit configured to acquire first information, the first information including environmental information of a terminal on which the first device is located; a communication unit configured to receive second information from a second device, the second information indicating regional information and / or time information of the terminal; a processing unit configured to output first sensing information for the terminal based on the first information and the second information; An apparatus comprising:
16. The acquisition unit includes: Detecting the environmental information of the terminal; or receiving the first information from the second device; 16. The apparatus of claim 15, specifically configured to:
17. The region information and / or the time information corresponds to a first algorithm, the first algorithm belonging to a predefined set of algorithms, and the processing unit: outputting the first sensing information for the terminal based on the first algorithm and the first information; 17. Apparatus according to claim 15 or 16, specifically adapted to:
18. The regional information and / or the temporal information are used to construct a first neural network model, and the processing unit: outputting the first sensing information for the terminal based on the first neural network model; 18. Apparatus according to any one of claims 15 to 17, specifically adapted to:
19. The communication unit includes: Transmitting the first sensing information to a fusion unit.
19. Apparatus according to any one of claims 15 to 18, specifically adapted to:
20. An information processing device, a communication unit configured to receive at least one portion of first information from at least one first device, the first information including environmental information of a terminal where the first device is located, and to receive second information from a second device, the second information indicating regional information and / or time information of the terminal; a processing unit configured to output first fusion information for the terminal based on at least a portion of the first information and the second information; An apparatus comprising:
21. The region information and / or the time information correspond to at least one first algorithm, the at least one first algorithm belonging to a predefined set of algorithms, and the processing unit: outputting the first fusion information for the terminal based on the at least one first algorithm and at least a portion of the first information; 21. Apparatus according to claim 20, specifically adapted to:
22. 22. The apparatus of claim 21, wherein multiple algorithms in the predefined algorithm set each correspond to a different sensor and / or correspond to a different region and / or time.
23. The regional information and / or the temporal information are used to construct a first neural network model, and the processing unit: Outputting the first fusion information for the terminal based on the first neural network model.
23. Apparatus according to any one of claims 20 to 22, specifically adapted to:
24. 24. The apparatus of claim 20, wherein the at least one portion of the first information comprises one or more of information from at least one of a sensor, a user, a communication module, or a map module.
25. The region information includes one or more selected from a continent, a country, a region, a state, or a city; and / or 25. The apparatus of claim 15, wherein the time information includes time system information, the time system information including one or more of seasons, holidays, solar terms, or seasonal timekeeping practices.
26. The first device comprises: Camera device, LiDAR, pickup device, ultrasonic sensor, or millimeter wave radar 26. Apparatus according to any one of claims 15 to 25, comprising any one of the following:
27. An information processing device, a processing unit configured to determine second information, the second information indicating regional information and / or time information of the terminal; a communication unit configured to transmit the second information to a first device or a third device; An apparatus comprising:
28. The processing unit includes: determining the second information based on a user input, the user input being a screen-based input or a voice input; or determining the second information based on an operator press; or determining the second information based on information from a map module; 28. The apparatus of claim 27, specifically configured to:
29. An information processing device comprising at least one processor and a communication interface, the communication interface being configured to receive signals from a communication device other than the information processing device and to transmit the signals to the at least one processor or to transmit signals from the at least one processor to a communication device other than the information processing device, the at least one processor being configured to implement, via logic circuits or by executing code instructions, a method according to any one of claims 1 to 5, or a method according to any one of claims 6 to 12, or a method according to any one of claims 13 and 14.
30. A terminal comprising a device according to any one of claims 15 to 19 and a device according to any one of claims 27 and 28, or A terminal comprising an apparatus according to any one of claims 20 to 26 and an apparatus according to any one of claims 27 and 28.
31. A computer-readable storage medium storing a computer program, the computer-readable storage medium causing, when the computer program is executed, the method according to any one of claims 1 to 5 to be implemented, or the method according to any one of claims 6 to 12 to be performed, or the method according to any one of claims 13 and 14 to be implemented.
32. A computer program product, the computer program product comprising computer programs or instructions which, when executed by a communication device, cause a method according to any one of claims 1 to 5 to be implemented, or a method according to any one of claims 6 to 12 to be implemented, or a method according to any one of claims 13 and 14 to be implemented.
Citation Information
Patent Citations
Automatic braking method and system based on image recognition and millimeter-wave radar fusion
CN107202983A
Time zone automatic setting method based on geographic position, mobile terminal and storage device
CN107332980A
Image information and radar information fusion method and system for traffic scene
CN108983219A
Vehicle control method and device, computer equipment and storage medium
CN111923919A
Vehicle perception method, device and system
CN112655226A
