Electronic device and method for driver assistance of vehicle

Abstract

The present disclosure relates to an electronic device and a method for driver assistance of a vehicle. An electronic device for driver assistance of a vehicle, the electronic device comprising a processing circuit, which is configured to: acquire first sensing data sensed by means of a sensor mounted on a vehicle; acquire second sensing data, wherein the second sensing data includes vehicle-to-everything (V2X) data from other devices that is used for indicating a surrounding environment of the vehicle; and determine a weight for performing sensor fusion by using the first sensing data and the second sensing data, so as to perform driver assistance of the vehicle on the basis of a result of the sensor fusion.

Description

Electronic devices and methods for driving assistance in vehicles

[0001] Priority Statement

[0002] This application claims priority to Chinese patent application No. 202411463624.7, filed on October 18, 2024, entitled "Electronic Device and Method for Driving Assistance in a Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of intelligent vehicles, and more specifically, to driver assistance systems for vehicles. Background Technology

[0004] In recent years, vehicle driver assistance technology has seen extensive development and research. Vehicle driver assistance technology involves the integration of sensor technology, image recognition, radar and lidar, in-vehicle communication, vehicle networking, and human-machine interaction technologies. These technologies work together to improve driving safety and the driving experience.

[0005] ADAS (Advanced Driver Assistance System) uses various sensors installed on the vehicle (such as cameras, radar, lasers, and ultrasonic sensors) to collect environmental data inside and outside the vehicle, and processes this data to identify, detect, and track the vehicle's surroundings, thereby assisting in driving and improving driving safety and comfort.

[0006] In addition, V2X (Vehicle to Everything) technology can also be used for vehicle driver assistance. V2X is a vehicle-to-everything wireless communication technology that enables vehicles to exchange information with their surroundings. Here, "V" represents the vehicle, and "X" represents roads, people, vehicles, equipment, and anything else that can be connected. By utilizing V2X technology to share information between the vehicle and its surroundings, it can be used to assist vehicle driving.

[0007] In addition, ISAC (Integrated Sensing and Communication) technology has received widespread attention in recent years and is considered one of the key technologies for future mobile communications. By integrating communication and sensing functions in the same system, ISAC enables efficient utilization of wireless signals, allowing the communication network to not only transmit information but also act as a giant sensor, better sensing and understanding the physical world through the transmission, reflection, and scattering of wireless signals. Summary of the Invention

[0008] A brief overview of this disclosure is given below to provide a basic understanding of some aspects of it. However, it should be understood that this overview is not an exhaustive summary of this disclosure. It is not intended to identify key or essential parts of this disclosure, nor is it intended to limit the scope of this disclosure. Its purpose is merely to present certain concepts of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0009] In vehicle driver assistance technologies (such as ADAS) known to the inventors, sensors mounted on the vehicle are typically used to collect environmental data to assist driving. However, the sensing data collected by the vehicle's sensors may be limited, for example, there may be blind spots, making it difficult to achieve a comprehensive perception of the vehicle's surroundings. On the other hand, if the ability of vehicle sensors to sense environmental data is improved by increasing the configuration of onboard sensors, there are problems such as increased vehicle manufacturing costs.

[0010] Therefore, it is considered to further acquire sensing data from other devices in the vehicle's surrounding environment (such as other vehicles, roadside equipment, etc.), which can be acquired using V2X-based acquisition methods and / or ISAC-based acquisition methods. This allows for sensor fusion, comprehensively utilizing sensing data collected by the vehicle's own sensors and sensing data collected by other devices in the surrounding environment for use in assisted driving. Consequently, driving assistance can be provided based on a more comprehensive understanding of the surrounding environment, improving driving safety, the effectiveness of driving decisions, and ultimately enhancing the performance of driving assistance systems.

[0011] According to one aspect of this disclosure, an electronic device for driving assistance in a vehicle is provided. The electronic device may include processing circuitry. The processing circuitry may be configured to: acquire first sensing data sensed by sensors mounted on the vehicle; acquire second sensing data, the second sensing data including vehicle-to-everything (V2X) data from other devices for indicating the vehicle's surrounding environment; and determine weights for sensor fusion using the first sensing data and the second sensing data to perform driving assistance in the vehicle based on the result of the sensor fusion.

[0012] According to another aspect of this disclosure, a method for driving assistance for a vehicle is provided. The method may include: acquiring first sensing data sensed by sensors mounted on the vehicle; acquiring second sensing data, the second sensing data including vehicle-to-everything (V2X) data from other devices for indicating the vehicle's surrounding environment; and determining weights for sensor fusion using the first sensing data and the second sensing data to perform driving assistance for the vehicle based on the result of the sensor fusion.

[0013] According to another aspect of this disclosure, a computer-readable storage medium is provided, including executable instructions that, when executed by an information processing device, cause the information processing device to perform a driving assistance method for a vehicle according to this disclosure.

[0014] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform a method for driving assistance for a vehicle according to this disclosure. Attached Figure Description

[0015] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0016] This disclosure will be more clearly understood with reference to the accompanying drawings and the following detailed description, wherein:

[0017] Figure 1 is a schematic diagram illustrating a driving assistance system according to an embodiment of the present disclosure;

[0018] Figure 2 is an exemplary configuration block diagram illustrating an electronic device for driving assistance in a vehicle according to an embodiment of the present disclosure;

[0019] Figure 3 is an exemplary flowchart illustrating a method for driving assistance for a vehicle according to an embodiment of the present disclosure;

[0020] Figure 4 is an exemplary flowchart illustrating a method for driving assistance for a vehicle according to another embodiment of the present disclosure;

[0021] Figure 5 is a schematic diagram illustrating an application scenario of driving assistance for a vehicle according to an embodiment of the present disclosure;

[0022] Figures 6-10 are exemplary signaling interaction diagrams illustrating driving assistance for a vehicle according to embodiments of the present disclosure; and

[0023] Figure 11 illustrates an exemplary configuration of a computing device that can implement an embodiment of the present invention. Detailed Implementation

[0024] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0025] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0026] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0027] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0028] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0030] Figure 1 is a schematic diagram illustrating a driver assistance system according to an embodiment of the present disclosure. The driver assistance system ADAS 100 is, for example, mounted on a vehicle for assisting in driving. As shown in Figure 1, ADAS 100 may include a sensor fusion module 116, a path prediction module 118, and a driving strategy determination module 120. Furthermore, for the sake of brevity, other modules in ADAS 100 are omitted, and only modules relevant to this disclosure are described.

[0031] The sensor fusion module 116 of ADAS 100 receives first sensing data 102 from the vehicle's internal sensors. In this disclosure, internal sensors refer to sensors mounted on the vehicle, such as, but not limited to, cameras, radar, and lidar mounted on the vehicle, and may also include various other sensors used to sense the vehicle's surrounding environment. The sensing data acquired from the internal sensors is referred to as "first sensing data." Figure 1 shows that the first sensing data 102 includes sensing data 104 from the camera, sensing data 106 from the radar, and sensing data 108 from the lidar.

[0032] In addition, ADAS 100 also receives second sensing data 110 from external sensors. In this disclosure, sensors not mounted on the vehicle but mounted on other devices (e.g., RSUs (Road Side Units), other vehicles, base stations, other traffic participants, etc.) are referred to as external sensors, and the sensing data acquired from these external sensors is referred to as second sensing data. As shown in FIG1, the second sensing data 110 may include V2X data 112 and ISAC signals 114.

[0033] Sensor fusion module 116 fuses the received first sensing data 102 and second sensing data 110, and inputs the result to path prediction module 118 for path prediction (e.g., predicting the vehicle's next travel route). Sensor fusion as described in this disclosure refers to the automatic analysis and synthesis of information from multiple sensors or data sources under certain criteria. Sensor fusion can include data-layer fusion, feature-layer fusion, and decision-layer fusion. Data-layer fusion (also known as pixel-level fusion) involves directly fusing the observation data from the sensors and then extracting feature vectors from the fused data for identification. Feature-layer fusion extracts representative features from the observation data provided by each sensor, fuses these features into a single feature vector, and then processes it. Decision-layer fusion is performed after each sensor has made a decision to improve the accuracy and reliability of the decision.

[0034] By fusing the first sensing data 102 and the second sensing data 110, path prediction can be performed by comprehensively considering the vehicle's own sensing data and the sensing data of other devices outside the vehicle, thus improving the accuracy of the predicted path and enhancing the performance of the driving assistance.

[0035] In addition, the driving strategy determination module 120 can determine the vehicle's next driving strategy (such as vehicle speed, steering wheel angle, throttle opening, brake pedal opening, etc.) based on the sensor fusion results and path prediction results. The determined driving strategy can be input to the vehicle's CAN bus 122 to assist and adjust the vehicle's driving behavior.

[0036] In addition, based on the determined driving strategy, driving warnings 124 can be generated (e.g., displayed on the vehicle's display screen or via voice prompts) to alert the driver. Driving warnings 124 may include alerts to the vehicle's surrounding environment (e.g., alerts to obstacles, traffic accidents, congestion, etc.) or prompts to the driver regarding driving strategies (e.g., adjusting vehicle speed, direction, etc.).

[0037] Next, the electronic device and method for driving assistance for a vehicle according to the present disclosure will be further described with reference to Figures 2 and 3.

[0038] Figure 2 illustrates an exemplary configuration block diagram of an electronic device 200 for driving assistance in a vehicle according to an embodiment of the present disclosure. The electronic device 200 can, for example, be used to implement the ADAS 100 shown in Figure 1.

[0039] In some embodiments, the electronic device 200 may include processing circuitry 210. The processing circuitry 210 of the electronic device 200 provides various functions of the electronic device 200.

[0040] Processing circuitry 210 can refer to various implementations of digital circuitry, analog circuitry, or mixed-signal (analog and digital combination) circuitry that perform functions in a computing system. Processing circuitry can include, for example, circuitry such as integrated circuits (ICs), application-specific integrated circuits (ASICs), portions or circuitry of a single processor core, an entire processor core, a single processor, programmable hardware devices such as field-programmable gate arrays (FPGAs), and / or systems comprising multiple processors.

[0041] In some embodiments, the processing circuit 210 may include a data acquisition unit 220 and a weight determination unit 230, configured to perform corresponding steps in the driving assistance method 300 for a vehicle shown in FIG3, described later.

[0042] In some embodiments, the electronic device 200 may further include a memory (not shown). The memory of the electronic device 200 may store information generated by the processing circuitry 210, as well as programs and data for the operation of the electronic device 210. The memory may be volatile memory and / or non-volatile memory. For example, the memory may include, but is not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), and flash memory. Furthermore, the electronic device 200 may be implemented at the chip level, or it may be implemented at the device level by including other external components.

[0043] Figure 3 shows an exemplary flowchart of a method 300 for driving assistance for a vehicle according to an embodiment of the present disclosure. This method 300 can, for example, be used in the electronic device 200 shown in Figure 2.

[0044] As shown in Figure 3, in step S310, the data acquisition unit 220 acquires first sensing data sensed by sensors mounted on the vehicle. This first sensing data may, for example, correspond to the first sensing data 102 in Figure 1. Additionally, in step S320, the data acquisition unit 220 acquires second sensing data, which includes V2X data from other devices used to indicate the vehicle's surrounding environment. This second sensing data may, for example, correspond to the second sensing data V2X data 112 in Figure 1.

[0045] V2X data from other devices may include data indicating basic information about the other devices, information indicating sensed objects around the other devices, and raw sensor data directly collected by the sensors of other devices. In some embodiments, V2X data may include V2X messages, such as BSM (Basic Safety Message), SSM (Sensor Sharing Message), and VIR (Vehicle Intention and Request) messages, to share relevant information in a predefined data format. In some embodiments, V2X data may include V2X raw data, i.e., raw sensor data directly collected by the sensors of other devices. Additionally, in some embodiments, feature data from the raw sensor data may also be obtained from other devices.

[0046] In some embodiments, V2X data can be acquired via cellular links and / or direct links. A cellular link is a communication link between a terminal device and a base station (hereinafter sometimes referred to as a uulink), suitable for large-volume data transmission between the vehicle and other devices such as the base station. For example, V2X raw data can be provided to the vehicle from other devices via a uulink. A direct link is a communication link between terminal devices (hereinafter sometimes referred to as a sidelink), suitable for data sharing between vehicles and between vehicles and roadside units (RSUs). For example, V2X messages can be provided to the vehicle from other devices via a sidelink.

[0047] Next, in step S320, the weight determination unit 230 determines the weights for sensor fusion using the first sensing data and the second sensing data, so as to provide driving assistance for the vehicle based on the result of sensor fusion. In this disclosure, driving assistance for the vehicle may include any operation that assists the driver in driving the vehicle, such as path prediction, driving strategy determination, and driving warning as described with reference to FIG1. ​​Additionally, driving assistance may also include autonomous driving that does not require or at least partially requires driver intervention.

[0048] In this disclosure, first sensing data collected by the vehicle's own sensors and second sensing data collected by other devices in the vehicle's surrounding environment are comprehensively utilized, and the weights of the first and second sensing data for sensor fusion are appropriately determined. This enables vehicle driving assistance based on a more comprehensive understanding of the surrounding environment, thereby improving driving safety, the effectiveness of driving decisions, and the performance of driving assistance.

[0049] In the above embodiments, V2X data from other devices was utilized as the second sensing data. However, in some cases, V2X communication (e.g., V2V (vehicle-to-vehicle) communication) may rely to some extent on direct communication within the line-of-sight (LOS) range. This means that communication quality can be affected by factors such as terrain, buildings, vehicle density, and weather, all of which can block or weaken the signal. Furthermore, V2X technology requires highly reliable and low-latency communication, which necessitates a highly stable and robust communication network. In deteriorating network conditions, V2X data transmission may be affected. Additionally, V2X technology requires vehicles and roadside infrastructure to possess powerful computing capabilities to process data in real time and make rapid decisions.

[0050] Therefore, in some embodiments, the second sensing data may further include an ISAC (Integrated Communication and Sensing) signal for sensing the vehicle's surrounding environment. ISAC technology can also sense the vehicle's surrounding environment to determine the position, speed, angle, etc., of objects within that environment. By integrating communication and sensing functions within the same system and sharing the same frequency band and hardware, ISAC can improve spectrum and energy efficiency. This integration reduces the need for additional hardware and can be implemented with lower complexity and cost. Furthermore, ISAC technology can provide beyond-line-of-sight sensing capabilities, enabling the detection of targets and obstacles beyond the vehicle's line of sight.

[0051] By combining V2X data with ISAC signals as the second sensing data, and performing sensor fusion together with the first sensing data sensed by the vehicle's internal sensors, and assigning appropriate fusion weights, the advantages of V2X data, ISAC signals, and the first sensing data obtained from the internal sensors can be complemented, resulting in a more comprehensive understanding of the vehicle's surrounding environment and further improving the performance of driver assistance systems. Additionally, in some cases, depending on the actual communication and vehicle driving environment, it is possible to choose to use only V2X data or ISAC signals as the second sensing data in sensor fusion. When using only ISAC signals as the second sensing data in sensor fusion, compared to using V2X data, communication and computational overhead can be saved and transmission efficiency improved.

[0052] Next, an exemplary flowchart of a method 400 for driving assistance for a vehicle according to another embodiment of the present disclosure will be described with reference to FIG4. This method may also be implemented, for example, by the electronic device 200 described in FIG2, and may also correspond to the method performed by the ADAS 100 according to the present disclosure as described in FIG1.

[0053] As shown in Figure 4, in step S402, sensor input is acquired, including first sensing data 430 and second sensing data 420.

[0054] In some embodiments, optionally, in step S404, the acquired second sensing data 420 is verified. Verification determines whether the second sensing data 420 is available for driving assistance of the vehicle.

[0055] In some embodiments, verification may include verifying the reliability of the second sensing data 420. For example, the confidence level of the second sensing data 420 may be obtained and compared with a preset confidence level threshold. If the confidence level of the second sensing data 420 is lower than the preset confidence level threshold, the second sensing data 420 is considered unreliable and may be discarded from subsequent sensor fusion. In some cases, the confidence level assigned to the second sensing data 420 may differ depending on its source. For example, when the second sensing data 420 is sensed by a sensor mounted on the RSU, a higher confidence level may be assigned, and this confidence level may be shared with the vehicle as part of the second sensing data. Conversely, when the second sensing data 420 is sensed by a sensor from another vehicle, a relatively lower confidence level may be assigned.

[0056] In some embodiments, verification may include verifying the sensor type of the second sensing data 420. If the sensor type sensing the second sensing data 420 is not the type required by the vehicle (e.g., the second sensing data 420 is sensed by LiDAR, and the vehicle's sensor fusion does not support the fusion of LiDAR with other types of data), it can be determined that the second sensing data 420 is unavailable for the vehicle's driving assistance. Alternatively, the sensor type of the second sensing data 420 may also be provided to the vehicle as part of the second sensing data 420 for verification.

[0057] In some embodiments, when verifying the second sensing data 420, the distance between the location of the sensing event sensed by the second sensing data 420 and the vehicle may be considered. For example, if the distance is far, the information carried by the second sensing data 420 may be of little help to the vehicle in understanding its surroundings, and thus it may be determined that the second sensing data 420 is unusable for the vehicle's driving assistance.

[0058] In some embodiments, the second sensing data 420 can also be verified using the first sensing data 430. For example, the confidence level of the second sensing data 420 can be determined by comparing the sensing events sensed by the first sensing data 430 with the sensing events sensed by the second sensing data 420. If the confidence level of the second sensing data 420 is lower than a preset confidence threshold, it can be determined that the second sensing data 420 is unavailable for driving assistance of the vehicle.

[0059] Next, in step S406, weights are assigned to the second sensing data 420 and the first sensing data 430, with weight w1 assigned to the second sensing data 420 and weight w2 assigned to the first sensing data 430. This step may correspond, for example, to step S330 in method 300 described with reference to FIG3.

[0060] In some embodiments, weighting can be assigned based on the data source of the second sensing data. For example, when the second sensing data is sensed by a sensor mounted on the RSU, it can be assigned a higher weight, while when the second sensing data is sensed by a sensor from another vehicle, it can be assigned a lower weight.

[0061] In some embodiments, weight allocation can be performed based on the confidence level of the second sensing data. This confidence level can be determined, for example, by reading the confidence information carried by the second sensing data. Alternatively, the confidence level can be determined through the verification step in step S404, by verifying the second sensing data 420 based on the first sensing data 430. In some embodiments, when the confidence level of the second sensing data 420 is high, the weight w1 of the second sensing data can be increased accordingly, thereby giving more consideration to the sensing results of the second sensing data 420 during sensor fusion.

[0062] In some embodiments, weight allocation can be based on the road environment in which the vehicle is traveling. The road environment can include different road scenarios such as highways, national / provincial roads, county / rural roads, and urban roads. Additionally, the road environment can also include traffic conditions related to the road, such as congestion, smooth traffic, and traffic accidents.

[0063] For example, in a highway environment where vehicles travel at high speeds, they may require more secondary sensor data from other devices to understand the surrounding environment over a larger area. Therefore, in this case, the weight w1 of the secondary sensor data can be increased accordingly. Conversely, in urban road environments where vehicles travel at lower speeds and have more traffic lights, the primary sensor data 430 may be more important for driver assistance; therefore, the weight w2 of the primary sensor data can be increased accordingly.

[0064] In addition, in road environments such as congestion, smooth traffic, and traffic accidents, the weights of the second sensing data 420 and the first sensing data 430 can be allocated according to the specific road environment to better adapt to the current road environment for driving assistance.

[0065] In some embodiments, weighting can be performed based on the distance of the location of the sensing event indicated by the second sensing data from the vehicle. In this disclosure, the sensing event indicated by the sensing data can refer to any event that can be determined based on the sensing data, including but not limited to sensing an obstacle, sensing a traffic accident, sensing a change in traffic conditions, sensing other events that may affect driving, etc.

[0066] For example, when the location of a sensing event is far from the vehicle, the first sensing data 430 may be insufficient to determine the sensing event, and the assistance of the second sensing data 420 may be required. In this case, the weight w1 of the second sensing data can be increased accordingly.

[0067] In some embodiments, weights can be assigned solely based on one of the following: the road environment in which the vehicle is traveling, the data source of the second sensing data, the confidence level of the second sensing data, and the distance from the location of the sensing event indicated by the second sensing data to the vehicle. In other embodiments, weights w1 and w2 can be assigned by considering at least two of the road environment, the confidence level of the second sensing data, and the distance from the location of the sensing event.

[0068] For example, in a road environment where a vehicle is driving on an urban road, if a traffic accident occurs at a distance from the vehicle, the weights w1 and w2 can be reasonably allocated by taking into account both the road environment and the distance of the sensed event, thereby achieving more optimized driving assistance.

[0069] Next, in step S408, sensor fusion is performed using the second sensing data 420 and the first sensing data 430 according to the weights w1 and w2 assigned in step S406. This allows the vehicle to gain a more comprehensive and reliable understanding of its surroundings.

[0070] Next, in step S410, the path prediction is performed using the sensor fusion results from step S408, which can provide the driver of the vehicle with a more comprehensive, accurate and optimized driving path plan.

[0071] Next, in some embodiments, in step S412, it is determined whether the sensing event indicated by the second sensing data 420 meets a predetermined criterion. This predetermined criterion may be, for example, a criterion that might affect the driving safety of the vehicle. In some embodiments, the predetermined criterion may include the distance between the location of the sensing event indicated by the second sensing data 420 and the vehicle being less than a predetermined distance threshold. For example, the sensing event is an obstacle ahead, and the obstacle is relatively close to the vehicle, below the predetermined distance threshold. In other embodiments, the predetermined criterion may include the degree of impact of the sensing event indicated by the second sensing data 420 on the driving of the vehicle being greater than a predetermined safety threshold. For example, the sensing event is a car accident occurring ahead of the lane in which the vehicle is traveling, and the degree of impact on the driving of the vehicle being greater than the safety threshold.

[0072] In some embodiments, in response to the sensing event indicated by the second sensing data 420 satisfying a predetermined criterion (determined as "yes" in step S412), the process proceeds to step S416, where a driving decision is made based on the sensor fusion result to determine the vehicle's driving strategy. The driving strategy may include determining the vehicle's speed, steering wheel angle, throttle opening, brake pedal opening, etc., thereby improving driving safety through driving assistance. In some embodiments, the driving strategy can be directly input into the vehicle's CAN bus (as shown in Figure 1), thereby automatically controlling the vehicle to drive according to the driving decision. In other embodiments, the driving decision information can also be presented to the driver through the vehicle's display screen, voice prompts, etc., so that the driver can adjust their driving style based on the driving decision information, reducing the possibility of danger.

[0073] In some embodiments, in response to the sensing event indicated by the second sensing data 420 not meeting a predetermined criterion (determined as "No" in step S412), the process proceeds to step S414. In this case, it is considered that the current sensing event has a minor impact on vehicle driving safety. Therefore, in step S414, a driving warning is issued to the vehicle driver based solely on the sensor fusion result, without participating in driving decisions or changing the driving strategy. Alternatively, in some embodiments, driving decisions and driving warnings can be performed simultaneously in step S416, providing the driver with feasible driving strategies while simultaneously alerting them. Furthermore, in some embodiments, the determination in step S412 may be omitted, and driving warnings and / or updates to the vehicle's driving strategy may be performed directly based on the sensor fusion result.

[0074] Next, the application scenarios and signaling interaction flow of vehicle driving assistance according to embodiments of the present disclosure will be described.

[0075] Figure 5 illustrates four application scenarios of vehicle driving assistance according to embodiments of the present disclosure. Figures 6-10 illustrate exemplary signaling interaction flows between devices in the corresponding scenarios.

[0076] Scenario 1 in Figure 5 corresponds to the signaling interaction diagram in Figure 6.

[0077] Scenario 1 includes a main vehicle 502, a roadside unit (RSU) 504, and a sensing target vehicle 500. The main vehicle 502 is equipped with the electronics 200 described above, which executes methods 300 or 400 described above to acquire first sensing data from the main vehicle 502 and second sensing data from the RSU 504 for sensor fusion. The RSU 504 corresponds to the other device (also referred to as the second electronics) in step S320 of method 300. Furthermore, the other device is not limited to the RSU and can also be other vehicles.

[0078] As shown in Figure 6, in S6000, RSU 504 broadcasts sensor sharing capabilities. This capability can be broadcast via sidelink through V2X messages. In some embodiments, the sensor sharing capability of RSU 504 can be broadcast via Sensor Sharing Message (SSM). SSM is a type of V2X message that allows vehicles to share raw perception data or processed environmental models from their sensors (such as radar, cameras, lidar, etc.), thereby enhancing the vehicle's perception of its surroundings through inter-vehicle information sharing.

[0079] The following shows an example structure of an SSM according to this disclosure: the “sourceType” field indicates the sensor type supported by the sender of the SSM, the “sensorSharingService” field indicates the type of second sensing data that can be provided, and the “detectedAbilityList” field indicates the ISAC sensing method that can be provided.

[0080] The specific data structure for the "SourceType" field is further illustrated below. "0" indicates a supported sensor type of camera ("Camera"), "1" indicates a supported sensor type of ultrasonic radar ("ultrasonicRadar"), "2" indicates a supported sensor type of millimeter-wave radar ("mmwaveRadar"), "3" indicates a supported sensor type of lidar ("Lidar"), "4" indicates support for cellular signals ("CellularSignal"), i.e., support for ISAC sensors, "5" indicates support for both camera and radar ("Camera&Radar"), "6" indicates support for both camera and lidar ("Camera&Lidar"), and "7" indicates support for both camera and ISAC sensors ("Camera&CellularSignal"). Additionally, the "SourceType" field can be configured to support other sensor types or combinations of multiple sensor types as needed.

[0081] The type of sensor supported by the device can be determined from the "SourceType" field in the SSM message. Additionally, for example, when the main vehicle 502 acquires second sensing data from RSU 504, by receiving the SSM message from RSU 504, it can be determined which type of sensor was used to measure the second sensing data (e.g., if the "SourceType" field in the RSU 504's SSM message is "3", it indicates that the second sensing data was measured using LiDAR).

[0082] The specific data structure for the "SensorSharingService" field is further illustrated below. "0" indicates support for message-based sensing sharing ("messageBasedSensingSharing"), meaning it supports sharing V2X messages with other devices. V2X messages can be any type of V2X message in a predefined format, such as indicating basic vehicle information including location, speed, and direction of travel, or information about sensed events. "1" indicates support for raw data-based sensing sharing ("rawDataBasedSensingSharing"), meaning it supports sharing V2X raw data with other devices. V2X raw data refers to the raw data directly collected by the sensor of the sending SSM, such as images captured by a camera or point cloud data generated by LiDAR. Such raw data can be directly shared with other devices to achieve more advanced collaborative perception and decision-making. For example, a vehicle can send LiDAR point cloud data to other vehicles to help them perceive occluded areas or improve the accuracy of their perception of the surrounding environment.

[0083] Additionally, in some embodiments, the "SensorSharingService" field, besides indicating V2X messages or V2X raw data, can also include a field indicating whether the type of sensed data is an ISAC signal. Furthermore, the ability to provide ISAC signals can also be determined through the "SourceType" field. For example, in the example data structure of "SourceType" above, "4" indicates support for ISAC sensors, and "7" indicates support for both cameras and ISAC sensors, both indicating the ability to provide sensed data of the ISAC signal type.

[0084] The specific data structure for the "detectedAbilityList" field is further illustrated below. Optionally, this field may include an indication of the available ISAC sensing methods (if ISAC sensing is supported). "MonostaticISAC" indicates monostatic sensing, where the transmitter and receiver are located at the same point on the same equipment or very close to each other. This means that radar signal transmission and reception are performed through the same antenna or adjacent antennas. "BistaticISAC" indicates bistatic sensing, where the transmitter and receiver are located on different equipment. This sensing method can provide a wider coverage area and can acquire target distance and velocity information from different angles, resulting in more accurate information, but the structure is relatively complex compared to monostatic sensing. "MultistaticISAC" indicates multistatic sensing, where multiple transmitters and receivers are distributed at different locations (e.g., different equipment). This configuration can further improve the system's coverage and enhance the accuracy of perception by acquiring target information from multiple angles.

[0085] Next, referring back to Figure 6, the host vehicle 502 acquires the SSM broadcast by the RSU 504. From this, the host vehicle 502 can determine the sensor sharing capability of the RSU 504. In some embodiments, the host vehicle 502 can request second sensing data from the RSU 504 based on the acquired SSM. For example, if the host vehicle 502 determines from the SSM from the RSU 504 that the RSU 504 supports providing V2X raw data, it can request the V2X raw data from the RSU 504 as the second sensing data. Alternatively, it can also request feature data of the V2X raw data. Furthermore, if the host vehicle 502 determines from the SSM from the RSU 504 that the sensor type used by the RSU 504 for sensing data is LiDAR, and the host vehicle 502 does not support sensor fusion for LiDAR, it may not request sensing data from the RSU 504.

[0086] Additionally, in some embodiments, in response to the main vehicle 502 determining, based on the SSM, that the RSU 504 possesses ISAC sensing capability, the main vehicle 502 can request the RSU 504 to share second sensing data based on ISAC. This allows sensing to be performed directly via ISAC, saving the SSM message passing process and thus conserving communication and computing resources as well as data decoding overhead.

[0087] In addition, in some embodiments, when the RSU 504 has ISAC sensing capability, the main vehicle 502 can further determine the ISAC sensing method that the RSU 504 can provide based on the SSM of the RSU 504, and request the RSU 504 to provide sensing data based on the corresponding sensing method.

[0088] In step S6002, the master vehicle 502 requests sensor sharing from the RSU 504. This sensor sharing request can be sent via a sidelink through a V2X message. In some embodiments, the sharing request can be sent via a VIR message. A VIR message is a type of V2X message that allows a vehicle to share its intentions, such as lane changes, acceleration, or deceleration, with other vehicles or roadside equipment, and can request specific assistance or information from other vehicles or roadside equipment, such as sharing sensor data.

[0089] In some embodiments, the request may include information indicating the type of the requested second sense data. For example, a "rawdataservice" field may be added to the VIR message to indicate whether sharing of raw data is required. Alternatively, a field indicating whether a V2X message or V2X raw data is requested may be set, corresponding to the SSM message.

[0090] Additionally, in some embodiments, where the RSU 504 has the capability to provide an ISAC sensing mode, the ISAC sensing mode of the RSU 504 can also be indicated via a sensor sharing request.

[0091] Next, in step S6004, RSU 504 senses the target vehicle 500. This sensing process can be seen, for example, in scenario 1 of Figure 5. RSU 504 sends an ISAC signal 506 through single-station sensing and receives the echo returned from the target vehicle 500, thereby sensing the target vehicle 500.

[0092] Next, in step S6008, RSU 504 transmits the sensed data (e.g., information indicating the position, angle, etc. of the sensed target vehicle 500) as a second sensed data transmission to the main vehicle 502. This transmission may be, for example, an SSM message 508 transmitted via a sidelink.

[0093] Thus, the main vehicle 502 obtains second sensing data from RSU 504, which can be used together with its own first sensing data for sensor fusion and driving assistance.

[0094] In some embodiments, optionally, in step S6002, the sensor sharing request sent by the master vehicle 502 to the RSU 504 may further include information indicating a sensing window. The sensing window may indicate the sensing time period and sensing range of the second sensing data desired by the master vehicle 502. In step S6006, the RSU 504 extracts the sensing data measured within the sensing time period and sensing range corresponding to the requested sensing window, and transmits it to the master vehicle 502 as the second sensing data in step S6008. Thus, the master vehicle 502 can request sensing data of the corresponding time period and range according to its own needs, thereby saving communication overhead and improving transmission efficiency.

[0095] The above description, with reference to Scenario 1 in FIG5 and FIG6, illustrates an application scenario of vehicle driver assistance according to an embodiment of the present disclosure and the corresponding signaling interaction flow. In Scenario 1 of FIG5, RSU 504 senses the target vehicle using the ISAC sensing method. It should be understood that the sensing method of RSU 504 is not limited to the ISAC sensing method, and can also be used to perform sensing using other sensors (cameras, radar, lidar, etc.). That is, the sensing step S6004 in FIG6 is not limited to the ISAC sensing method, but can be other types of sensing methods, and can be shared with the master vehicle 502 through V2X data (V2X messages or V2X raw data). In addition, the master vehicle 502 can specify the sensing method of RSU 504 through a request (e.g., a VIR message) in step S6002, based on the sensor sharing capability SSM broadcast by RSU 504.

[0096] Next, referring to Scenario 2 in FIG5 and FIG7, another application scenario of vehicle driving assistance according to an embodiment of the present disclosure and the corresponding signaling interaction flow will be described.

[0097] Scenario 2 includes a master vehicle 510, a requesting vehicle 514 (also referred to as a fourth electronic device), and a target vehicle 512. The master vehicle 514 may be equipped with the electronic device 200 described above, executing methods 300 or 400 described above to acquire first and second sensing data for sensor fusion. In this scenario, the master vehicle 510 actively senses the target vehicle 512. Additionally, the requesting vehicle 514 requests sensor data sharing from the master vehicle 510, and the master vehicle 510 can provide the information of the target vehicle 512 as second sensing data to the requesting vehicle 514. Furthermore, the requesting vehicle 514 may also be equipped with the electronic device 200 described above, executing methods 300 or 400 described above to perform sensor fusion based on the first sensing data sensed by its own sensors and the second sensing data acquired from the master vehicle 510.

[0098] As shown in Figure 7, in step S7000, the requesting vehicle 514 requests sensor sharing from the master vehicle. This step is similar to the request step described with reference to step S6002 in Figure 6, and will not be repeated here. In step S7002, the master vehicle 510 broadcasts its sensor sharing capability. This broadcasting step is similar to the broadcasting step described with reference to step S6000 in Figure 6, and will not be repeated here. Furthermore, it should be understood that the order of steps S7000 and S7002 can be interchanged; that is, the master vehicle 510 can first broadcast its own sensor sharing capability SSM, and the requesting vehicle 514, after receiving the SSM broadcast by the master vehicle 510, then requests sensor sharing from the master vehicle 510 based on that SSM.

[0099] Next, in step S7004, the main vehicle 510 senses the target vehicle 512. This sensing process can be seen, for example, in scenario 2 of Figure 5. The main vehicle 510 sends an ISAC 516 through single-station sensing and receives the echo returned from the target vehicle 512, thereby sensing the target vehicle 512.

[0100] Next, in step S7008, the master vehicle 510 transmits the sensed data (e.g., information indicating the position, angle, etc. of the sensing target vehicle 512) as a second sense data transmission to the requesting vehicle 514. This transmission may be, for example, an SSM message 518 transmitted via a sidelink.

[0101] Therefore, the requesting vehicle 514 obtains the second sensing data from the host vehicle 510, which can be used together with its own sensed first sensing data for sensor fusion and driving assistance. Additionally, the host vehicle 510 can also utilize the sensed information from the target vehicle 512 for sensor fusion and driving assistance.

[0102] In some embodiments, optionally, in step S7000, the sensor sharing request sent by the requesting vehicle 514 to the master vehicle 510 may further include information for indicating a sensing window. The sensing window may indicate the sensing time period and sensing range of the second sensing data desired by the requesting vehicle 514. In step S7006, the master vehicle 510 extracts the sensing data measured within the sensing time period and sensing range corresponding to the requested sensing window, and transmits it as the second sensing data to the requesting vehicle 514 in step S7008. Thus, the requesting vehicle 514 can request sensing data of the corresponding time period and range according to its own needs, thereby saving communication overhead and improving transmission efficiency.

[0103] Furthermore, the sensing steps in step S7004 of Figure 7, similar to step S6004 in Figure 6, are not limited to the ISAC sensing method (ISAC 516) shown in scenario 2 of Figure 5. They can be other types of sensing methods, or sensing using other sensors (cameras, radar, lidar, etc.). In other words, the sensing steps in step S7004 of Figure 7 are not limited to the ISAC sensing method; they can be other types of sensing methods and shared with the requesting vehicle 514 via V2X data (V2X messages or V2X raw data). Additionally, the requesting vehicle 514 can, based on the sensor sharing capability SSM broadcast by the master vehicle 510, share the corresponding sensing method with the master vehicle 510 (e.g., via VIR messages).

[0104] The scenarios 1 and 2 in Figure 5 described above are based on ISAC single-station sensing. Next, we will describe the scenarios based on ISAC multi-station sensing with reference to scenarios 3 and 4 in Figure 5.

[0105] Scenario 3 in Figure 5 and Figures 8-9 illustrate another application scenario of driving assistance for a vehicle according to an embodiment of the present disclosure and the corresponding signaling interaction flow.

[0106] Scenario 3 includes a main vehicle 520, an RSU 522, and a sensing target vehicle 524. The main vehicle 520 may be equipped with the electronic device 200 described above, which executes the methods 300 or 400 described above to acquire first sensing data and second sensing data from the RSU 522 for sensor fusion. The RSU 522 corresponds to the other device (also referred to as the third electronic device) in step S320 of method 300. Furthermore, the other device is not limited to the RSU and may also be other vehicles.

[0107] As shown in Figure 8, in step S8000, the master vehicle 520 requests sensor sharing from the RSU 522. This step is similar to step S6002 described with reference to Figure 6, and will not be repeated here. In step S8002, the master vehicle 520 broadcasts its sensor sharing capability. This broadcasting step is similar to the broadcasting step described with reference to step S6000 in Figure 6, and will not be repeated here. Furthermore, it should be understood that the order of steps S8000 and S8002 can be interchanged; that is, the master vehicle 520 can first broadcast its own sensor sharing capability SSM, and then request sensor sharing from the RSU 522.

[0108] Next, in step S8004, RSU 522 senses the target vehicle 524. This sensing process can be seen, for example, in scenario 3 of Figure 5. RSU 522 senses the target vehicle 524 via bi-station sensing (ISAC 530) (i.e., the transmitting end of the ISAC signal 530 is RSU 522, it passes through the target vehicle 524, and the receiving end is the main vehicle 520), and the sensing signal ISAC 530 is directly transmitted to the main vehicle 520. In some embodiments, RSU 522 can provide second sensing data to the main vehicle 520 based on the Sensor Sharing Capability (SSM) broadcast by the main vehicle 520 in step S8002. For example, RSU 522 can determine whether the master vehicle 520 has ISAC bi-station sensing capability based on the SSM broadcast by the master vehicle 520. If it has this capability, RSU 522 can directly provide the second sensing data to the master vehicle 520 through ISAC bi-station sensing without transmitting SSM messages, thereby saving communication and computing resources as well as data decoding overhead.

[0109] Next, in step S8008, the main vehicle 520 broadcasts the received sensing data. For example, the sensing data can be broadcast via an SSM message. Additionally, in some embodiments, the SSM message can specify the type of the sensing data as an ISAC signal, and the ISAC sensing method as bi-station sensing. Thus, other vehicles 528 or other roadside equipment (not shown) can also receive the sensing data for their own sensor fusion, etc.

[0110] Alternatively, in step S8006, the main vehicle 520 may trim a request sensing window to broadcast the sensing data measured within the corresponding sensing time period and sensing range in step S8008. This sensing window can be determined by the main vehicle 520 itself based on the situation.

[0111] In the signaling interaction flow shown in Figure 8, vehicle 528 passively receives sensing data (SSM 534) broadcast from master vehicle 520 as the receiver. Additionally, the sensing target vehicle 524 can also receive this sensing data (SSM 532).

[0112] Next, referring to Figure 9, another signaling interaction process corresponding to scenario 3 is described. The difference between this signaling interaction process and that in Figure 8 is that vehicle 528, as the requesting vehicle, requests sensor sharing from master vehicle 520. In response to the request from requesting vehicle 528, master vehicle 520 requests sensor sharing from RSU 522.

[0113] Specifically, in step S9000, the requesting vehicle 528 (also referred to as the fourth electronic device) requests sensor sharing from the master vehicle 520. This step is similar to the request step described with reference to step S6002 in FIG6, and will not be repeated here. In step S9002, the master vehicle 520 broadcasts its sensor sharing capability. This broadcasting step is similar to the broadcasting step described with reference to step S6000 in FIG6, and will not be repeated here. In addition, it should be understood that the order of steps S9000 and S9002 can be interchanged, that is, the master vehicle 520 can first broadcast its own sensor sharing capability SSM, and the requesting vehicle 528 can then request sensor sharing from the master vehicle 520 based on the SSM after receiving the master vehicle 520's SSM.

[0114] Next, in step S9004, RSU 522 performs sensing on the target vehicle 524. This step is similar to the sensing step S8004 described with reference to FIG8, and also uses the ISAC dual-station sensing method.

[0115] Next, in step S9008, the master vehicle 520 transmits the received sensing data as a second sensing data to the requesting vehicle 528. This step is similar to the transmission step described with reference to step S7008 in FIG7, and will not be repeated here.

[0116] In some embodiments, optionally, in step S9000, the sensor sharing request sent by the requesting vehicle 528 to the master vehicle 510 may further include information for indicating a sensing window. The sensing window may indicate the sensing time period and sensing range of the second sensing data desired by the requesting vehicle 528. In step S9006, the master vehicle 520 extracts the sensing data measured within the sensing time period and sensing range corresponding to the requested sensing window, and transmits it as the second sensing data to the requesting vehicle 528 in step S9008. Thus, the requesting vehicle 528 can request sensing data of the corresponding time period and range according to its own needs, thereby saving communication overhead and improving transmission efficiency.

[0117] Furthermore, it should be understood that the sensing method of RSU 522 is not limited to the ISAC sensing method; it can also be sensing using other sensors (cameras, radar, lidar, etc.). That is, the sensing methods in step S8004 of Figure 8 and step S9004 of Figure 9 are not limited to the ISAC sensing method; they can be other types of sensing methods and shared with the host vehicle 520 via V2X data (V2X messages or V2X raw data). Additionally, in Figure 9, the requesting vehicle 528 can specify the sensing method of RSU 522 through a request (e.g., a VIR message) in step S9000, based on the sensor sharing capability SSM broadcast by the host vehicle 520.

[0118] Next, referring to Scenario 4 in FIG5 and FIG10, another application scenario of vehicle driving assistance according to an embodiment of the present disclosure and the corresponding signaling interaction flow will be described. The difference between this application scenario and Scenario 3 is that the device initiating ISAC dual-station sensing is ISAC vehicle 538, rather than RSU 522 in Scenario 3.

[0119] Steps S1000 and S1002 in Figure 10 correspond to steps S9000 and S9002 in Figure 9, respectively, and will not be repeated here. Next, we will only describe the differences between this scenario and scenario 3.

[0120] In step S1004, the ISAC vehicle 538 senses the target vehicle 542. This sensing process can be illustrated in scenario 4 of Figure 5, where the ISAC vehicle 538 senses the target vehicle 542 via bi-station sensing (ISAC 544) (i.e., the ISAC signal 544 is transmitted from the ISAC vehicle 538 to the target vehicle 542, and then received by the main vehicle 536), and the sensing signal ISAC 544 is directly transmitted to the main vehicle 536. In some embodiments, the ISAC vehicle 538 can determine whether the main vehicle 536 has ISAC bi-station sensing capability through the sensor sharing capability (SSM) broadcast by the main vehicle 536 in step S1002. If it has this capability, the ISAC vehicle 538 can directly provide the second sensing data to the main vehicle 536 via ISAC bi-station sensing without transmitting an SSM message, thereby saving communication resources and data decoding overhead. Additionally, in some embodiments, the ISAC vehicle 538 may also broadcast its own sensor-sharing capabilities, thereby alerting other vehicles or roadside equipment, for example, to its ISAC sensing capabilities.

[0121] In step S1008, the master vehicle 536 transmits the received sensing data as second sensing data (SSM 550) to the requesting vehicle 540. Alternatively, the master vehicle 536 may also crop the sensing window in step S1006. These steps are similar to those described with reference to Figure 9 and will not be repeated here.

[0122] Additionally, Figure 10 illustrates the signaling interaction flow for sensor sharing in response to a request from vehicle 540. In some embodiments, for scenario 4, similar to Figure 8, the master vehicle 536 may spontaneously (e.g., periodically, or following predetermined criteria) perform sensor sharing and broadcast the sensing results to other devices. For example, as shown in scenario 4 of Figure 5, other vehicles 540 may receive the broadcast sensing data SSM 550, the sensing target vehicle 542 may receive the broadcast sensing data SSM 546, and the ISAC vehicle 538 may receive the broadcast sensing data SSM 548.

[0123] Furthermore, although scenario 4 in Figure 5 shows vehicle 538 as an ISAC vehicle, it should be understood that the sensing method of vehicle 538 is not limited to ISAC sensing; it can also be sensing using other sensors (cameras, radar, lidar, etc.). That is, the sensing method in step S1004 of Figure 10 is not limited to ISAC sensing; it can be other types of sensing methods and shared with the master vehicle 536 via V2X data (V2X messages or V2X raw data). Additionally, in Figure 10, the requesting vehicle 540 can specify the sensing method of vehicle 538 through a request (e.g., a VIR message) in step S1000, based on the sensor sharing capability SSM broadcast by the master vehicle 536.

[0124] The application scenarios and signaling interaction flows of vehicle driving assistance according to embodiments of the present disclosure have been described above with reference to Figures 5-10. It should be understood that the above scenarios and signaling interaction flows are merely examples, and those skilled in the art can design other application scenarios and signaling interaction flows according to actual needs.

[0125] Figure 11 illustrates an exemplary configuration that enables the implementation of a computing device 1100 according to an embodiment of the present invention.

[0126] Computing device 1100 is an example of a hardware device capable of applying the above aspects of the present invention. Computing device 1100 can be any machine configured to perform processing and / or computation. Computing device 1100 can be, but is not limited to, a workstation, server, desktop computer, laptop computer, tablet computer, personal data assistant (PDA), wireless terminal, drone, portable smart device, vehicle terminal, Internet of Things device, or a combination thereof.

[0127] As shown in Figure 11, computing device 1100 may include one or more components that can be connected to or communicate with bus 1102 via one or more interfaces. Bus 1102 may include, but is not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus. Computing device 1100 may include, for example, one or more processors 1104, one or more input devices 1106, and one or more output devices 1108. The one or more processors 1104 may be any type of processor and may include, but is not limited to, one or more general-purpose processors or dedicated processors (such as dedicated processing chips). Processor 1102 may, for example, correspond to processor 210 in Figure 2 and be configured to implement the functions of various units of the electronic device for driving assistance in a vehicle disclosed herein. Input device 1106 may be any type of input device capable of inputting information to the computing device and may include, but is not limited to, a mouse, keyboard, touchscreen, microphone, and / or remote controller. The output device 1108 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator and / or printer.

[0128] The computing device 1100 may also include or be connected to a non-transitory storage device 1114, which may be any non-transitory storage device capable of storing data, and may include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, compressed disks or any other optical media, cache memory and / or any other storage chip or module, and / or any other medium from which a computer may read data, instructions and / or code. The computing device 1100 may also include random access memory (RAM) 1110 and read-only memory (ROM) 1112. ROM 1112 may store executable programs, utilities, or processes in a non-volatile manner. RAM 1110 provides volatile data storage and stores instructions related to the operation of the computing device 1100. The computing device 1100 may also include a network / bus interface 1116 coupled to a data link 1118. The network / bus interface 1116 can be any kind of device or system capable of enabling communication with external devices and / or networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices and / or chipsets (such as Bluetooth™ devices, IEEE 802.11 devices, WiFi devices, WiMax devices, mobile cellular communication facilities, etc.).

[0129] It should be understood that the reference to "embodiment" or similar expressions in this specification means that a specific feature, structure, or characteristic described in connection with that embodiment is included in at least one specific embodiment of this disclosure. Therefore, the appearance of the terms "in embodiments of this disclosure" and similar expressions in this specification does not necessarily refer to the same embodiment.

[0130] Those skilled in the art will understand that this disclosure can be implemented as a system, apparatus, method, or as a computer-readable storage medium (e.g., a non-transient storage medium) as a computer program product. Therefore, this disclosure can be implemented in various forms, such as a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microprogram code, etc.), or a software and hardware embodiment, hereinafter referred to as a "circuit," "module," or "system." Furthermore, this disclosure can also be implemented as a computer program product in any tangible media form, having computer-usable program code stored thereon.

[0131] The description herein is based on flowcharts and / or block diagrams of systems, apparatuses, methods, and computer program products according to specific embodiments of this disclosure. It will be understood that each block in each flowchart and / or block diagram, and any combination of blocks in the flowcharts and / or block diagrams, can be implemented using computer program instructions. These computer program instructions are executable by a machine comprising a processor of a general-purpose computer or a special-purpose computer, or other programmable data processing means, and are processed by the computer or other programmable data processing means to perform the functions or operations described in the flowcharts and / or block diagrams.

[0132] The accompanying drawings illustrate flowcharts and block diagrams showing the architecture, functionality, and operation of systems, apparatuses, methods, and computer program products achievable according to various embodiments of the present disclosure. Thus, each block in a flowchart or block diagram may represent a module, segment, or portion of program code, including one or more executable instructions to implement a specified logical function. It should also be noted that in some other embodiments, the functions described in a block may not be performed in the order shown in the figures. For example, two blocks illustrated as connected may actually be executed simultaneously, or in some cases, depending on the functions involved, they may be executed in the reverse order shown in the figures. Furthermore, it should be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware, or by a combination of dedicated hardware and computer instructions, to perform specific functions or operations.

[0133] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to market technology of the embodiments, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0134] Note that the technology disclosed in this specification may have the following configurations.

[0135] (1) An electronic device for driving assistance in a vehicle, the electronic device comprising:

[0136] Processing circuit, the processing circuit being configured to:

[0137] Acquire first sensing data from sensors mounted on the vehicle;

[0138] Acquire second sensing data from other devices, the second sensing data including vehicle-to-everything (V2X) data for indicating the vehicle's surrounding environment; and

[0139] The weights for sensor fusion using the first sensing data and the second sensing data are determined to provide driving assistance for the vehicle based on the result of sensor fusion.

[0140] (2) The electronic device according to (1), wherein,

[0141] The second sensing data also includes a communication-sensing integrated ISAC signal for sensing the vehicle's surrounding environment.

[0142] (3) The electronic device according to (1), wherein determining the weights for sensor fusion using the first sensing data and the second sensing data includes:

[0143] The weights are determined based on one or more of the following: the road environment in which the vehicle is traveling, the data source of the second sensing data, the confidence level of the second sensing data, and the distance of the location of the sensing event indicated by the second sensing data from the vehicle.

[0144] (4) The electronic device according to (3), wherein the confidence level of the second sensing data is determined by verifying the second sensing data with the first sensing data.

[0145] (5) The electronic device according to (1), wherein the processing circuit is further configured to:

[0146] In response to the sensing event indicated by the second sensing data satisfying a predetermined criterion, a driving strategy for the vehicle is determined based on the result of sensor fusion.

[0147] (6) The electronic device according to (5), wherein the predetermined criteria include:

[0148] The location of the sensing event indicated by the second sensing data is less than a predetermined distance threshold from the vehicle; and / or

[0149] The impact of the sensing event indicated by the second sensing data on the driving of the vehicle is greater than a predetermined safety threshold.

[0150] (7) The electronic device according to (1), wherein the processing circuit is further configured to:

[0151] Based on the results of sensor fusion, the vehicle is given driving warnings and / or its driving strategy is updated.

[0152] (8) The electronic device according to (1), wherein the other device includes a second electronic device, and the processing circuit is further configured to:

[0153] Acquire a Sensor Sharing Message (SSM) indicating the sensor information sharing capability of the second electronic device; and

[0154] The second electronic device requests second sensing data based on the acquired SSM.

[0155] (9) The electronic device according to (8), wherein the SSM includes one or more of the following fields:

[0156] A field used to indicate the supported sensor types;

[0157] A field used to indicate the type of second sensing data that can be provided;

[0158] A field used to indicate the available ISAC sensing methods.

[0159] (10) The electronic device according to (9), wherein,

[0160] Supported sensor types include cameras, radar, lidar, and ISAC sensors.

[0161] The second type of sensing data includes ISAC signals, V2X messages, and V2X source data.

[0162] ISAC sensing methods include single-station sensing, dual-station sensing, and multi-station sensing.

[0163] (11) The electronic device according to (8), wherein requesting the second sensing data from the second electronic device based on the acquired SSM includes:

[0164] In response to determining, based on the SSM, that the second electronic device has ISAC sensing capability, a request is made for the second electronic device to share second sensing data based on ISAC.

[0165] (12) The electronic device according to (1), wherein the other device includes a third electronic device, and the processing circuit is further configured to:

[0166] Broadcasting a Sensor Sharing Message (SSM) to indicate the vehicle's sensor information sharing capability; and

[0167] Receive second sensing data provided to the vehicle according to the SSM from a third electronic device.

[0168] (13) The electronic device according to (1), wherein the processing circuit is further configured to:

[0169] Receive a request for second sensing data from a fourth electronic device; and

[0170] In response to the request, the second sensing data is shared with the fourth device.

[0171] (14) The electronic device according to (11) or (13), wherein the request includes information for indicating a sensing window, the sensing window indicating the sensing time period and sensing range of the desired second sensing data.

[0172] (15) The electronic device according to (11) or (13), wherein the request includes information indicating the type of the requested second sensing data.

[0173] (16) A method for driving assistance for a vehicle, the method comprising:

[0174] Acquire first sensing data from sensors mounted on the vehicle;

[0175] Acquire second sensing data from other devices, the second sensing data including vehicle-to-everything (V2X) data for indicating the vehicle's surrounding environment; and

[0176] The weights for sensor fusion using the first sensing data and the second sensing data are determined to provide driving assistance for the vehicle based on the result of sensor fusion.

[0177] (17) According to the method described in (16), wherein,

[0178] The second sensing data also includes a communication-sensing integrated ISAC signal for sensing the vehicle's surrounding environment.

[0179] (18) According to the method of (16), wherein determining the weights for sensor fusion using the first sensing data and the second sensing data includes:

[0180] The weights are determined based on one or more of the following: the road environment in which the vehicle is traveling, the data source of the second sensing data, the confidence level of the second sensing data, and the distance of the location of the sensing event indicated by the second sensing data from the vehicle.

[0181] (19) According to the method of (18), wherein the confidence level of the second sensing data is determined by verifying the second sensing data with the first sensing data.

[0182] (20) The method according to (16) further includes:

[0183] In response to the sensing event indicated by the second sensing data satisfying a predetermined criterion, a driving strategy for the vehicle is determined based on the result of sensor fusion.

[0184] (21) The method according to (20), wherein the predetermined criteria include:

[0185] The location of the sensing event indicated by the second sensing data is less than a predetermined distance threshold from the vehicle; and / or

[0186] The impact of the sensing event indicated by the second sensing data on the driving of the vehicle is greater than a predetermined safety threshold.

[0187] (22) The method according to (16) further includes:

[0188] Based on the results of sensor fusion, the vehicle is given driving warnings and / or its driving strategy is updated.

[0189] (23) The method according to (16), wherein the other device includes a second electronic device, and the method further includes:

[0190] Acquire a Sensor Sharing Message (SSM) indicating the sensor information sharing capability of the second electronic device; and

[0191] The second electronic device requests second sensing data based on the acquired SSM.

[0192] (24) According to the method of (23), wherein the SSM includes one or more of the following fields:

[0193] A field used to indicate the supported sensor types;

[0194] A field used to indicate the type of second sensing data that can be provided;

[0195] A field used to indicate the available ISAC sensing methods.

[0196] (25) According to the method described in (24), wherein,

[0197] Supported sensor types include cameras, radar, lidar, and ISAC sensors.

[0198] The second type of sensing data includes ISAC signals, V2X messages, and V2X source data.

[0199] ISAC sensing methods include single-station sensing, dual-station sensing, and multi-station sensing.

[0200] (26) The method according to (23), wherein requesting the second sensing data from the second electronic device based on the acquired SSM includes:

[0201] In response to determining, based on the SSM, that the second electronic device has ISAC sensing capability, a request is made for the second electronic device to share second sensing data based on ISAC.

[0202] (27) The method according to (16), wherein the other device includes a third electronic device, and the method further includes:

[0203] Broadcasting a Sensor Sharing Message (SSM) to indicate the vehicle's sensor information sharing capability; and

[0204] Receive second sensing data provided to the vehicle according to the SSM from a third electronic device.

[0205] (28) The method according to (16) further includes:

[0206] Receive a request for second sensing data from a fourth electronic device; and

[0207] In response to the request, the second sensing data is shared with the fourth electronic device.

[0208] (29) The method according to (26) or (28), wherein the request includes information for indicating a sensing window, the sensing window indicating the sensing time period and sensing range of the desired second sensing data.

[0209] (30) The method according to (26) or (28), wherein the request includes information indicating the type of the requested second sensing data.

[0210] (31) A computer-readable storage medium including executable instructions that, when executed by an information processing apparatus, cause the information processing apparatus to perform the method described in any one of (16) to (30).

[0211] (32) A computer program product comprising a computer program that, when run by a processor, causes the processor to perform the method described in any one of (16) to (30).

Claims

1. An electronic device for driving assistance in a vehicle, the electronic device comprising: Processing circuit, the processing circuit being configured to: Acquire first sensing data sensed by sensors mounted on the vehicle; Acquire second sensing data from other devices, the second sensing data including vehicle-to-everything (V2X) data for indicating the vehicle's surrounding environment; as well as Determine the weights for sensor fusion using the first sensing data and the second sensing data to provide driving assistance for the vehicle based on the sensor fusion result.

2. The electronic device according to claim 1, wherein, The second sensing data also includes a communication-sensing integrated (ISAC) signal for sensing the vehicle's surrounding environment. 3.The electronic device of claim 1, wherein, Determining the weights for sensor fusion using the first sensing data and the second sensing data includes: The weights are determined based on one or more of the following: the road environment in which the vehicle is traveling, the data source of the second sensing data, the confidence level of the second sensing data, and the distance of the location of the sensing event indicated by the second sensing data from the vehicle.

4. The electronic device of claim 3, wherein, The confidence level of the second sensing data is determined by verifying the second sensing data using the first sensing data. 5.The electronic device of claim 1, wherein, The processing circuit is further configured to: In response to the sensing event indicated by the second sensing data satisfying a predetermined criterion, a driving strategy for the vehicle is determined based on the result of sensor fusion.

6. The electronic device of claim 5, wherein, The predetermined criteria include: The location of the sensing event indicated by the second sensing data is less than a predetermined distance threshold from the vehicle; and / or The impact of the sensing event indicated by the second sensing data on the driving of the vehicle is greater than a predetermined safety threshold. 7.The electronic device of claim 1, wherein, The processing circuit is further configured to: Based on the results of sensor fusion, the vehicle is given driving warnings and / or its driving strategy is updated. 8.The electronic device of claim 1, wherein, The other devices include a second electronic device, and the processing circuit is further configured to: Acquire a Sensor Sharing Message (SSM) indicating the sensor information sharing capability of the second electronic device; and The second electronic device requests second sensing data based on the acquired SSM.

9. The electronic device of claim 8, wherein, The SSM includes one or more of the following fields: A field used to indicate the supported sensor types; A field used to indicate the type of second sensing data that can be provided; A field used to indicate the available ISAC sensing methods.

10. The electronic device according to claim 9, wherein, Supported sensor types include cameras, radar, lidar, and ISAC sensors. The second type of sensing data includes ISAC signals, V2X messages, and V2X source data. ISAC sensing methods include single-station sensing, dual-station sensing, and multi-station sensing. 11.The electronic device of claim 8, wherein The request for second sensing data from the second electronic device based on the acquired SSM includes: In response to determining, based on the SSM, that the second electronic device has ISAC sensing capability, a request is made for the second electronic device to share second sensing data based on ISAC.

12. The electronic device of claim 1, wherein, The other devices include a third electronic device, and the processing circuit is further configured to: Broadcast Sensor Sharing Message (SSM) to indicate the vehicle's sensor information sharing capability; as well as Receive second sensing data provided to the vehicle according to the SSM from a third electronic device.

13. The electronic device of claim 1, wherein, The processing circuit is further configured to: Receive a request for second sensing data from a fourth electronic device; and In response to the request, the second sensing data is shared with the fourth device.

14. The electronic device of claim 11 or 13, wherein, The request includes information for indicating a sensing window, which indicates the sensing time period and sensing range of the desired second sensing data.

15. The electronic device of claim 11 or 13, wherein, The request includes information indicating the type of the requested second sensing data.

16. A method for driving assistance in a vehicle, the method comprising: Acquire first sensing data from sensors mounted on the vehicle; Acquire second sensing data from other devices, the second sensing data including vehicle-to-everything (V2X) data for indicating the vehicle's surrounding environment; as well as The weights for sensor fusion using the first sensing data and the second sensing data are determined to provide driving assistance for the vehicle based on the result of sensor fusion.

17. The method according to claim 16, wherein, The second sensing data also includes a communication-sensing integrated ISAC signal for sensing the vehicle's surrounding environment.

18. The method of claim 16, wherein, Determining the weights for sensor fusion using the first sensing data and the second sensing data includes: The weights are determined based on one or more of the following: the road environment in which the vehicle is traveling, the data source of the second sensing data, the confidence level of the second sensing data, and the distance of the location of the sensing event indicated by the second sensing data from the vehicle.

19. The method of claim 18, wherein, The confidence level of the second sensing data is determined by verifying the second sensing data using the first sensing data.

20. The method of claim 16, further comprising: In response to the sensing event indicated by the second sensing data satisfying a predetermined criterion, a driving strategy for the vehicle is determined based on the result of sensor fusion.

21. The method of claim 20, wherein, The predetermined criteria include: The location of the sensing event indicated by the second sensing data is less than a predetermined distance threshold from the vehicle; and / or The impact of the sensing event indicated by the second sensing data on the driving of the vehicle is greater than a predetermined safety threshold.

22. The method of claim 16, further comprising: Based on the results of sensor fusion, the vehicle is given driving warnings and / or its driving strategy is updated.

23. The method of claim 16, wherein, The other devices include a second electronic device, and the method further includes: Acquire a Sensor Sharing Message (SSM) indicating the sensor information sharing capability of the second electronic device; and The second electronic device requests second sensing data based on the acquired SSM.

24. The method of claim 23, wherein, The SSM includes one or more of the following fields: A field used to indicate the supported sensor types; A field used to indicate the type of second sensing data that can be provided; A field used to indicate the available ISAC sensing methods.

25. The method according to claim 24, wherein, Supported sensor types include cameras, radar, lidar, and ISAC sensors. The second type of sensing data includes ISAC signals, V2X messages, and V2X source data. ISAC sensing methods include single-station sensing, dual-station sensing, and multi-station sensing.

26. The method of claim 23, wherein, The request for second sensing data from the second electronic device based on the acquired SSM includes: In response to determining, based on the SSM, that the second electronic device has ISAC sensing capability, a request is made for the second electronic device to share second sensing data based on ISAC.

27. The method of claim 16, wherein, The other devices include a third electronic device, and the method further includes: Broadcasting a Sensor Sharing Message (SSM) to indicate the vehicle's sensor information sharing capability; and Receive second sensing data provided to the vehicle according to the SSM from a third electronic device.

28. The method of claim 16, further comprising: Receive a request for second sensing data from a fourth electronic device; as well as In response to the request, the second sensing data is shared with the fourth electronic device.

29. The method of claim 26 or 28, wherein, The request includes information for indicating a sensing window, which indicates the sensing time period and sensing range of the desired second sensing data.

30. The method of claim 26 or 28, wherein, The request includes information indicating the type of the requested second sensing data.

31. A computer-readable storage medium comprising executable instructions that, when executed by an information processing apparatus, cause the information processing apparatus to perform the method of any one of claims 16 to 30.

32. A computer program product comprising a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 16 to 30.

Images