Integrated traffic condition acquisition communication equipment and method

Abstract

The invention provides an integrated traffic condition acquisition communication device and method, and relates to the technical field of intelligent traffic, road traffic participant information based on a Raywhile fusion data flow is obtained, a vehicle obtains traffic environment information associated with the vehicle through V2X communication, the road traffic participant information and the traffic environment information are subjected to data fusion, and the traffic condition information is acquired. Road traffic condition information is obtained and uploaded to the cloud server, and the road traffic condition information is sent to a vehicle end through the cloud server. The modules are connected with the SOC chip instead of a switch network, so that time delay caused by network transmission is avoided while the number of devices is reduced, and the system is suitable for large-scale deployment and maintenance. Through acquisition of the millimeter-wave radar, the short-focus camera and the long-focus camera and communication by using the cellular network, the optical fiber network and the V2X, thunder-vision fusion can be realized in a larger range, so that more comprehensive and accurate road traffic condition information is obtained.

Description

Technical Field

[0001] The present invention relates to the field of intelligent traffic technology, and in particular to an integrated traffic condition collection and communication device; in addition, the present invention also relates to an integrated traffic condition collection and communication method. Background Art

[0002] Vehicle-road-cloud integration is an intelligent transportation system that leverages advanced wireless communications and next-generation internet technologies to enable dynamic, real-time information exchange among vehicles, roads, and the cloud. This integration not only addresses the limitations of single-vehicle intelligence and propels autonomous driving toward fully autonomous driving, but also integrates with smart cities to build efficient and safe transportation systems. Examples include traffic management and optimization, automatic detection of traffic accidents, road obstructions, and rapid response. Roadside equipment, as the infrastructure for vehicle-road-cloud integration, is developing towards more comprehensive perception, more efficient communication, and more intelligent decision-making.

[0003] However, after years of development, a model has basically been formed that includes communication equipment, perception equipment, and computing equipment. The independent split solution for each device has problems such as too many devices, high prices, and poor real-time performance, which makes it difficult to meet the needs of large-scale deployment. Summary of the Invention

[0004] To address the challenges of existing technologies, at least one embodiment of the present invention provides an integrated traffic condition collection and communication device. This reduces the number of devices, reduces data transmission latency, and provides more comprehensive and accurate road traffic condition information, making it suitable for large-scale deployment and maintenance. To this end, at least one embodiment of the present invention also provides a traffic condition collection and communication method.

[0005] In a first aspect, the present invention provides an integrated traffic condition collection and communication device, comprising: SOC chip, used for data calculation and data transmission; Millimeter-wave radar, connected to the SOC chip, is used to locate road traffic participants; The short-focus camera is connected to the SOC chip and is used to collect close-range road traffic image information; A telephoto camera, connected to the SOC chip, is used to collect long-distance road traffic image information; The V2X communication module is connected to the SOC chip and is used to communicate with the vehicle; Cellular communication module, connected to the SOC chip, used to communicate with the cloud server; The network port is connected to the SOC chip and is used to communicate with external devices or cloud servers.

[0006] Preferably, the present invention provides an integrated traffic condition collection and communication device, wherein the SOC chip includes: A radar-vision fusion unit, which detects and tracks road traffic participants using millimeter-wave radar, short-focus cameras, and long-focus cameras; The V2X receiving unit is used to analyze traffic environment information related to the vehicle side; a cellular receiving unit for parsing received cloud server communication information; A network receiving unit for parsing communication information from external devices or cloud servers; Multi-source data fusion unit, used to fuse data to obtain road traffic condition information; A routing unit, used to obtain information received by the vehicle through a network port or a cellular communication module; The data sending unit is used to send road traffic condition information to the vehicle end.

[0007] Preferably, in the integrated traffic condition collection and communication device provided by the present invention, the focal length of the short-focus camera is 12 mm and is tilted downward by 12 degrees.

[0008] Preferably, in the integrated traffic condition collection and communication device provided by the present invention, the focal length of the telephoto camera is 25 mm or 50 mm, and is tilted downward by 3 degrees or 2 degrees.

[0009] Preferably, the integrated traffic condition collection and communication device provided by the present invention further includes: The positioning module is connected to the SOC chip and is used for device positioning and timing.

[0010] In a second aspect, the present invention further provides an integrated traffic condition collection and communication method, which uses the integrated traffic condition collection and communication device of the first aspect, comprising: Obtain road traffic participant information based on radar and visual fusion data stream; Vehicles obtain relevant traffic environment information through V2X communication; Fusing road traffic participant information and traffic environment information to obtain road traffic condition information and upload it to the cloud server; The road traffic condition information is sent to the vehicle via the cloud server.

[0011] Preferably, the present invention provides an integrated traffic condition collection and communication method for obtaining road traffic participant information based on a radar-visual fusion data stream, comprising: Target detection and tracking of road traffic participants using millimeter-wave radar, short-focus camera, and long-focus camera; Collect structured data from millimeter-wave radar, short-focus camera, and long-focus camera at the same time; Using the millimeter-wave radar data as a benchmark, project the short-focus camera data and the long-focus camera data into the millimeter-wave radar's coordinate system. The millimeter-wave radar data, short-focus camera data, and long-focus camera data of the same target are clustered in the millimeter-wave radar coordinate system and the structured information is fused.

[0012] Preferably, the integrated traffic condition collection and communication method provided by the present invention clusters millimeter-wave radar data, short-focus camera data, and long-focus camera data of the same target in the coordinate system of the millimeter-wave radar and fuses structured information, including: The clustered data is used as measurement values ​​and input into the tracker for filtering.

[0013] Preferably, the integrated traffic condition collection and communication method provided by the present invention further includes: The target detection model is used to detect and identify occlusion and traffic queue phenomena.

[0014] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any one of the methods of the first aspect when executed by a processor.

[0015] As can be seen, the integrated traffic condition collection and communication device and method provided by this invention connects each module to a SOC chip rather than through a switch network, thus reducing the number of devices while avoiding the time delay caused by network transmission, making it suitable for large-scale deployment and maintenance. By collecting information using millimeter-wave radar, short-focus cameras, and long-focus cameras, and communicating using cellular networks, fiber optic networks, and V2X, radar and visual fusion can be achieved over a wider range, thereby obtaining more comprehensive and accurate road traffic condition information. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 Shown is a schematic diagram of the framework of an integrated traffic condition collection and communication device according to an embodiment of the present invention; Figure 2 Shown is a schematic diagram of the composition of the SOC chip in an embodiment of the present invention; Figure 3 Shown is a flow chart of an integrated traffic condition collection and communication method according to an embodiment of the present invention; Figure 4Shown is a flowchart of radar-visual fusion in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0019] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. In this document, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0020] Example 1 In the existing technology, after years of development, a model has basically been formed that includes communication equipment, sensing equipment, and computing equipment. The independent split solution for each device has problems such as numerous devices, high prices, and poor real-time performance, which makes it difficult to meet the needs of large-scale deployment. Example 1 of the present invention provides the following solution: like Figure 1-2 As shown, this embodiment provides an integrated traffic condition collection and communication device, including: The SOC chip is used for data processing and transmission. It should be noted that the SOC chip can receive data in real time and perform real-time calculations, generating comprehensive perception data that is transmitted to the vehicle and cloud servers. Furthermore, each component is connected via the external pins of the computing unit SOC chip, rather than through a switch network, to avoid time delays caused by network transmission.

[0021] Millimeter-wave radar, connected to the SOC chip, is used to locate road traffic participants. It should be noted that the millimeter-wave radar is connected to the SOC chip via the CAN bus. Millimeter-wave radars of corresponding specifications are selected according to different application scenarios. Specifically, in urban intersection scenarios, millimeter-wave radars with a lateral coverage range of 12 lanes and a longitudinal coverage range of 350 meters are selected; in highway scenarios, millimeter-wave radars with a lateral coverage range of 10 lanes and a longitudinal coverage range of 500 meters or 1000 meters are selected.

[0022] The short-focus camera is connected to the SOC chip and is used to collect close-range road traffic image information. It should be noted that the short-focus camera is connected to the SOC chip via MIPI.

[0023] In some embodiments, in both urban intersection scenarios and highway scenarios, the focal length of the short-focus camera can be 12 mm and tilted downward by 12 degrees.

[0024] The telephoto camera is connected to the SoC chip and is used to collect long-distance road traffic image information. It should be noted that the telephoto camera is connected to the SoC chip via MIPI.

[0025] In some embodiments, in an urban intersection scenario, the telephoto camera has a focal length of 25 mm and is tilted downward by 3 degrees. In a highway scenario, the telephoto camera has a focal length of 50 mm and is tilted downward by 2 degrees.

[0026] It is understandable that the monitoring ranges of the short-focus camera and the long-focus camera should overlap to ensure that the target can be monitored by both the short-focus and long-focus cameras simultaneously within a certain position range. The farthest monitoring range of the long-focus camera should include the farthest detection distance of the millimeter-wave radar to ensure that the camera data can optimize the millimeter-wave radar data throughout the coverage range of the millimeter-wave radar.

[0027] The V2X communication module is connected to the SOC chip and is used to communicate with the vehicle. It should be noted that the V2X communication module is connected to the SOC chip via USB.

[0028] In some embodiments, the V2X communication module may use LTE-V2X, a vehicle network wireless communication technology designed based on 4G, or NR-V2X, a vehicle network wireless communication technology designed based on 5G.

[0029] The cellular communication module is connected to the SOC chip and is used to communicate with the cloud server. It should be noted that the cellular communication module is connected to the SOC chip via USB.

[0030] In some embodiments, when a wired network is unavailable, the cellular communication module can use 5G, and can upload the perception data of the all-in-one computer to the cloud through 5G communication technology. It can also obtain information such as the overall traffic situation and road conditions from the cloud, such as road closure construction, road changes, weather conditions and other data.

[0031] The network port is connected to the SoC chip and is used to communicate with external devices or cloud servers. It should be noted that the network port is connected to the SoC chip via RMII.

[0032] Specifically, the network port is connected to the traffic light at the intersection via a wired network, such as optical fiber. Without considering factors such as the light's physical appearance, the system directly captures background data, including the light's phase and countdown timer, along with the location of lane lines and stop lines at the intersection. This data helps identify dangerous situations, such as pedestrians crossing the road when the light is red.

[0033] In some embodiments, the device further includes a positioning module, which is connected to the SOC chip and is used for device positioning and timing.

[0034] It should be noted that the positioning module is connected to the SOC chip via a UART serial port. In some embodiments, the positioning module adopts a Beidou positioning component.

[0035] In some embodiments, the SOC chip includes: The radar and vision fusion unit is used to detect and track road traffic participants through millimeter-wave radar, short-focus camera and long-focus camera.

[0036] It should be noted that the radar-visual fusion unit is connected to the millimeter-wave radar, short-focus camera, and long-focus camera. It identifies traffic targets based on short-focus and long-focus images, and fuses them with the millimeter-wave radar detection results to obtain radar-visual fusion data.

[0037] The V2X receiving unit is used to analyze traffic environment information related to the vehicle. It should be noted that the V2X receiving unit is connected to the communication module and analyzes the position and speed information of passing vehicles installed with the on-board unit (OBU).

[0038] The cellular receiving unit is used to parse the received cloud server communication information. It should be noted that the cellular receiving unit is connected to the cellular communication module. In some embodiments, the cellular receiving unit adopts a 5G receiving unit.

[0039] The network receiving unit is used to analyze the communication information of the external device or the cloud server. It should be noted that the network receiving unit is connected to the network interface.

[0040] The multi-source data fusion unit is used to fuse data to obtain road traffic condition information. It should be noted that the multi-source data fusion unit is connected to the radar and vision fusion unit, the V2X receiving unit, the cellular receiving unit, and the network receiving unit.

[0041] The routing unit is used to obtain information received by the vehicle through the network port or cellular communication module. It should be noted that the routing unit directly obtains information that the terminal needs to publish directly to the vehicle through the network port and cellular communication module.

[0042] The data transmission unit is used to transmit road and traffic condition information to the vehicle. It should be noted that the data transmission unit is connected to the multi-source data fusion unit and transmits the final fused data to the cloud server via a network port or cellular communication module. Traffic condition information, road condition information, and information that the cloud needs to release directly to the vehicle are then distributed to the vehicle via the V2X communication module.

[0043] In the implementation scenario 1, a vehicle equipped with an OBU is traveling straight ahead at high speed in front of a stationary target vehicle, pedestrian or non-motor vehicle that is not equipped with an OBU. The device can detect the dangerous target and notify the vehicle equipped with the OBU to slow down in time to avoid a collision.

[0044] In the second implementation scenario, when there are obstacles blocking the view of vehicles, vehicles equipped with OBU and vehicles, pedestrians or non-motorized vehicles not equipped with OBU traveling in a straight or perpendicular path at the intersection, the device can detect dangerous targets and notify the vehicles equipped with OBU to avoid collisions.

[0045] In implementation scenario three, the device obtains traffic light phase and countdown information and broadcasts it to vehicles within 500 meters of the road section through V2X wireless communication. The relevant vehicles calculate a reasonable speed based on their own status to ensure that they do not run red lights or green waves.

[0046] Example 2 like Figure 3 As shown, this embodiment provides an integrated traffic condition collection and communication method, which uses the integrated traffic condition collection and communication device in Example 1, including: Step 1: Obtain road traffic participant information based on the radar-visual fusion data stream.

[0047] It's important to note that road traffic participant information includes information on motor vehicles, non-motor vehicles, pedestrians, and other traffic participants. Given the limited penetration of connected vehicle technology and the inability of non-motor vehicles and pedestrians to carry V2X communication modules, it's impossible to obtain all traffic participant information solely through V2X communication modules. Therefore, proactive traffic condition sensing through roadside sensors is crucial. The combination of millimeter-wave radar and cameras offers the optimal balance of performance, sensing range, and cost, enabling wider monitoring and greater trajectory continuity for targets within the monitored area.

[0048] In some embodiments, as Figure 4 The method of obtaining the radar-visual fusion data stream includes: Step 1.1: Detect and track road traffic participants using millimeter-wave radar, short-focus camera, and long-focus camera.

[0049] It should be noted that the position and speed information of each traffic participant is obtained from the millimeter radar, and the image data of the short-focus camera and the long-focus camera are subjected to image detection, target tracking and vehicle attribute recognition, such as license plate and body color, to obtain the position and attributes of each traffic participant in the pixel coordinate system.

[0050] Step 1.2: Collect structured data from the millimeter-wave radar, short-focus camera, and long-focus camera at the same time.

[0051] In step 1.3, the short-focus camera data and the long-focus camera data are projected into the coordinate system of the millimeter-wave radar based on the millimeter-wave radar data.

[0052] In step 1.4, the millimeter-wave radar data, short-focus camera data, and long-focus camera data of the same target are clustered in the millimeter-wave radar coordinate system and the structured information is fused.

[0053] In some embodiments, cluster data is used as a measurement value and input into a tracker for filtering. The filtering may be a Kalman filter or a particle filter, etc., thereby increasing the accuracy of the cluster data.

[0054] In some embodiments, occlusion and traffic queue phenomena are detected and identified through a target detection model, and the target detection model can be trained using a YOLO model.

[0055] Step 2: The vehicle obtains traffic environment information related to it through V2X communication.

[0056] It's important to note that vehicle-related traffic environment information includes information about traffic between other vehicles, between vehicles and road infrastructure, between vehicles and people, and between vehicles and the network. Due to the complexity of traffic scenarios, digital twins derived from active roadside perception cannot accurately locate the movement trajectory of every traffic participant in all scenarios. Vehicle information obtained through V2X communication modules compensates for this deficiency. Roadside equipment information, such as traffic light status, is obtained through network receiving units, while cloud-based information is obtained through fiber optic networks or 5G.

[0057] Step 3: Fuse the road traffic participant information and traffic environment information to obtain road traffic condition information and upload it to the cloud server.

[0058] It should be noted that through covariance interaction fusion, more comprehensive final fusion data can be obtained, and traffic condition information and road condition information can be analyzed.

[0059] Step 4: Send the road traffic condition information to the vehicle via the cloud server.

[0060] Example 3 This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method of embodiment 1 of the present invention are implemented.

[0061] For example, machine-readable storage media may include, but are not limited to, various known and unknown types of non-volatile memory.

[0062] In summary, Examples 1-3 of the present invention provide an integrated traffic condition collection and communication device and method. Each module is connected to a system-on-chip (SOC) chip rather than through a switch network, thereby reducing the number of devices while avoiding the time delays caused by network transmission, making it suitable for large-scale deployment and maintenance. By collecting information using millimeter-wave radar, short-focus cameras, and long-focus cameras, and communicating using cellular networks, fiber optic networks, and V2X, radar and visual fusion can be achieved over a wider range, thereby obtaining more comprehensive and accurate road traffic condition information.

[0063] It will be understood by those skilled in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0064] In the embodiments of the present application, the disclosed systems, devices, and methods can be implemented in other ways. For example, the division of units is merely a logical functional division, and other division methods can be used in actual implementation. For example, multiple units or components can be combined or integrated into another system. In addition, the coupling between the various units can be direct coupling or indirect coupling. In addition, the various functional units in the embodiments of the present application can be integrated into a processing unit or can exist separately physically.

[0065] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0066] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a machine-readable storage medium. Therefore, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a machine-readable storage medium, which can include several instructions for enabling an electronic device to execute all or part of the process of the technical solution described in the embodiments of the present application. The above-mentioned storage medium can include various media that can store program codes, such as ROM, RAM, removable disk, hard disk, magnetic disk or optical disk.

[0067] The above content is only a specific embodiment of the present application, and the protection scope of the present application is not limited thereto. Those skilled in the art may make changes or substitutions within the technical scope disclosed in the present application, and these changes or substitutions should all be within the protection scope of the present application.

Claims

1. An integrated traffic condition collection and communication device, characterized in that: include: SOC chip, used for data calculation and data transmission; A millimeter-wave radar, connected to the SOC chip, for locating road traffic participants; A short-focus camera, connected to the SOC chip, for collecting short-range road traffic image information; A telephoto camera, connected to the SOC chip, for collecting long-distance road traffic image information; A V2X communication module, connected to the SOC chip, for communicating with the vehicle; A cellular communication module, connected to the SOC chip, for communicating with the cloud server; The network port is connected to the SOC chip and is used to communicate with external devices or cloud servers.

2. The integrated traffic condition collection and communication device according to claim 1, characterized in that: The SOC chip includes: A radar and vision fusion unit, configured to detect and track road traffic participants using the millimeter-wave radar, short-focus camera, and long-focus camera; The V2X receiving unit is used to analyze traffic environment information related to the vehicle side; a cellular receiving unit for parsing received cloud server communication information; A network receiving unit for parsing communication information from external devices or cloud servers; Multi-source data fusion unit, used to fuse data to obtain road traffic condition information; a routing unit, configured to obtain information received by the vehicle through the network port or the cellular communication module; The data sending unit is used to send road traffic condition information to the vehicle end.

3. The integrated traffic condition collection and communication device according to claim 1, characterized in that: The short-throw camera has a focal length of 12 mm and is tilted downward 12 degrees.

4. The integrated traffic condition collection and communication device according to claim 1, characterized in that: The telephoto camera has a focal length of 25 mm or 50 mm and is tilted downward by 3 degrees or 2 degrees.

5. The integrated traffic condition collection and communication device according to claim 1, characterized in that: Also includes: The positioning module is connected to the SOC chip and is used for device positioning and timing.

6. An integrated traffic condition collection and communication method, characterized in that: The integrated traffic condition collection and communication device according to any one of claims 1 to 5 is applied, comprising: Obtain road traffic participant information based on radar and visual fusion data stream; Vehicles obtain relevant traffic environment information through V2X communication; Fusing the road traffic participant information and traffic environment information to obtain road traffic condition information and uploading it to a cloud server; The road traffic condition information is sent to the vehicle end via the cloud server.

7. The integrated traffic condition collection and communication method according to claim 6, characterized in that: The obtaining of road traffic participant information based on the radar-visual fusion data stream includes: Target detection and tracking of road traffic participants using millimeter-wave radar, short-focus camera, and long-focus camera; Collecting structured data from the millimeter-wave radar, short-focus camera, and long-focus camera at the same time; Projecting the short-focus camera data and the long-focus camera data into the coordinate system of the millimeter-wave radar based on the millimeter-wave radar data; The millimeter-wave radar data, the short-focus camera data, and the long-focus camera data of the same target are clustered in the coordinate system of the millimeter-wave radar and the structured information is fused.

8. The integrated traffic condition collection and communication method according to claim 7, characterized in that: The clustering of the millimeter-wave radar data, the short-focus camera data, and the long-focus camera data of the same target in the coordinate system of the millimeter-wave radar and fusing structured information includes: The clustered data is used as measurement values ​​and input into the tracker for filtering.

9. The integrated traffic condition collection and communication method according to claim 6, characterized in that: Also includes: The target detection model is used to detect and identify occlusion and traffic queue phenomena.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 6 to 9 are implemented.

Images