An internet of vehicles deep learning visual auxiliary positioning method

Abstract

The application discloses a kind of Internet of Vehicles deep learning visual auxiliary positioning method, it is related to communication technical field, it solves the technical problem that existing technology has data acquisition blind area, leading to the positioning data of acquisition is insufficient, cannot satisfy the accurate positioning of vehicle under full scene;Cloud platform is set in the application, after receiving positioning information, positioning information is judged and supplemented;When positioning information is insufficient to form complete positioning simulation image, it is supplemented by auxiliary equipment and obtained, combined with multi-source data, the positioning information is supplemented, the problem of data acquisition blind area is solved, to ensure that the application can satisfy the accurate positioning of vehicle under full scene;The cloud platform of the application sends positioning simulation image to target vehicle, and sends positioning information to the Internet of Vehicles communication equipment of adjacent vehicle;From large area to small area, both vehicle and obstacle are timely and accurately positioned.

Description

Technical Field

[0001] The present invention belongs to the field of communications and relates to a vision-assisted positioning technology based on deep learning, specifically a deep learning vision-assisted positioning method for vehicle networks. Background Art

[0002] In vehicle-to-vehicle communication technology, positioning information is transmitted between different vehicles or between a vehicle and other devices. Other vehicles or devices use the positioning information to obtain the vehicle's current location. Therefore, vehicle positioning technology based on the vehicle-to-vehicle network is of great significance to vehicle safety, traffic control, etc.

[0003] In the existing technology, the Internet of Vehicles communication device generates vehicle positioning information, and other Internet of Vehicles communication devices analyze the position information in the vehicle positioning information to determine the vehicle position; the existing technology avoids positioning errors caused by vehicle model and size by determining the positional relationship between the positioning unit in the vehicle and the vehicle positioning reference point; however, when the existing technology uses visual positioning for vehicle positioning, there will be blind spots in data collection, resulting in insufficient positioning data and unable to meet the requirements of accurate vehicle positioning in all scenarios; therefore, there is an urgent need for a visually assisted positioning method that can be used in all scenarios. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a deep learning vision-assisted positioning method for the Internet of Vehicles, which is used to solve the technical problem that when the prior art has blind spots in data collection, the collected positioning data is insufficient and cannot meet the requirements of accurate vehicle positioning in all scenarios. After collecting and obtaining the positioning information, the present invention analyzes and judges the positioning information. When the positioning information is missing, it is supplemented by auxiliary equipment to solve the above-mentioned problem.

[0005] To achieve the above objectives, according to an embodiment of the first aspect of the present invention, a method for deep learning vision-assisted positioning in an Internet of Vehicles (IoV) is proposed, comprising:

[0006] The positioning unit acquires positioning information of the first vehicle in conjunction with the image acquisition device, and transmits the positioning information to the cloud platform and adjacent vehicles via the vehicle network communication device; wherein the positioning information includes offset information of the obstacle relative to the positioning unit position, the positioning unit position, and offset information of the positioning unit position relative to the positioning reference point of the first vehicle;

[0007] The cloud platform judges and supplements the positioning information, generates a positioning simulation image based on the positioning information, and sends the positioning simulation image to the Internet of Vehicles communication equipment of other vehicles; wherein, the positioning simulation image is obtained by real-time rendering based on the positioning information.

[0008] Preferably, obtaining positioning information through the image acquisition device includes:

[0009] Acquire a real-time image of the vehicle through an image acquisition device provided on the first vehicle; wherein the image acquisition device includes a camera and a driving recorder;

[0010] The positioning unit extracts and analyzes the pre-processed real-time vehicle image through a machine learning algorithm and generates positioning information; the pre-processing includes image segmentation, image denoising and grayscale transformation.

[0011] Preferably, the positioning reference point is the geometric center of the first vehicle.

[0012] Preferably, the position of the positioning unit is specifically the longitude and latitude of the positioning unit.

[0013] Preferably, the positioning unit includes a positioning bracket and a processor; wherein the positioning bracket is used to install the image acquisition device, and the processor communicates and / or is electrically connected with the image acquisition device and the vehicle network communication device respectively.

[0014] Preferably, obtaining the offset information of the obstacle by the positioning unit includes:

[0015] Establish a three-dimensional rectangular coordinate system with the reference point of the image acquisition device as the origin, and mark it as the first coordinate system;

[0016] Extract obstacles from the real-time image of the vehicle through a machine learning algorithm, calculate the coordinates of the obstacles in the first coordinate system, and mark them as the first coordinates;

[0017] A three-dimensional rectangular coordinate system is established with the positioning unit position as the origin and marked as the second coordinate system. The coordinate system transformation method combines the first coordinates to obtain the coordinates of the obstacle in the second coordinate system and mark them as the second coordinates. The second coordinates are the offset information of the obstacle relative to the positioning unit position.

[0018] Preferably, the cloud platform analyzes and supplements the positioning information, including:

[0019] The cloud platform divides the positioning information according to the position of the first vehicle and generates five pieces of position information;

[0020] When the five position information are continuous and not empty, it is determined that the positioning information does not need to be supplemented; otherwise, the empty position information is supplemented by the auxiliary equipment.

[0021] Preferably, the cloud platform sends the positioning simulation image to the target vehicle; wherein the target vehicle is a vehicle that has been started in the target area, and the target area is drawn based on the positioning reference point of the first vehicle.

[0022] Preferably, acquiring the target area includes:

[0023] A circular area is drawn with the positioning reference point of the first vehicle as the center and the set length as the radius, and is marked as the target area; or

[0024] A rectangular area is drawn with the positioning reference point of the first vehicle as the center, the road width as the set width, and the set length, and marked as the target area.

[0025] Preferably, when the positioning information is sent to the cloud platform, the positioning information is also sent to the Internet of Vehicles communication equipment of the adjacent vehicle. The adjacent vehicle determines the position of the first vehicle and the position of the obstacle based on the positioning information of the first vehicle.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. The present invention sets up a cloud platform. After receiving the positioning information, the cloud platform judges and supplements the positioning information. When the positioning information is insufficient to form a complete positioning simulation image, it is supplemented by auxiliary equipment and combined with multi-source data to supplement the positioning information, thus solving the problem of data collection blind spots and ensuring that the present invention can meet the requirements of vehicle precise positioning in all scenarios.

[0028] 2. The cloud platform of the present invention sends a positioning simulation image to the target vehicle. The target vehicle determines and verifies its own position and the position of the obstacle based on the positioning simulation image. When sending the positioning information to the cloud platform, the positioning information is also sent to the vehicle network communication equipment of the adjacent vehicle. The adjacent vehicle determines the position of the first vehicle and the position of the obstacle based on the positioning information of the first vehicle; from large areas to small areas, timely and accurate positioning of vehicles and obstacles is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the working steps of the present invention. DETAILED DESCRIPTION

[0030] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] When assisting vehicle positioning based on deep convolution and machine vision, many scenarios will be affected by missing data, resulting in insufficient positioning data, and it will be impossible to accurately position the vehicle in all scenarios. After collecting and obtaining positioning information, this application analyzes and judges the positioning information. When the positioning information is missing, it is supplemented by auxiliary equipment to solve the problem of narrow applicability of existing scenarios.

[0032] See also Figure 1 The present invention provides a deep learning vision-assisted positioning method for an Internet of Vehicles, comprising:

[0033] The positioning unit obtains the positioning information of the first vehicle in combination with the image acquisition device, and sends the positioning information to the cloud platform and adjacent vehicles respectively through the vehicle network communication device;

[0034] The cloud platform judges and supplements the positioning information, generates a positioning simulation image based on the positioning information, and sends the positioning simulation image to the Internet of Vehicles communication equipment of other vehicles.

[0035] In a specific embodiment, the positioning information includes offset information of the obstacle relative to the positioning unit position, the positioning unit position, and offset information of the positioning unit position relative to the first vehicle positioning reference point.

[0036] In an optional embodiment, the positioning reference point is the geometric center of the first vehicle; in other preferred embodiments, the positioning reference point is the intersection of the standard rectangle and the longitudinal center line of the first vehicle, or the positioning reference point is the intersection of the standard rectangle and the transverse center line of the first vehicle; it is worth noting that the reference rectangle is a rectangle formed by line segments of the outer contour of the first vehicle.

[0037] In an optional embodiment, the obstacles are specifically objects that may cause damage to the vehicle, such as height limit poles, stone piers, guardrails, pedestrians in the middle of the road, etc. Detection and identification of obstacles is very necessary, and through the exchange of obstacle information between vehicles, traffic accidents can be effectively avoided.

[0038] In a specific embodiment, the positioning unit includes a positioning bracket and a processor. The positioning bracket is used to install an image acquisition device. The image acquisition device can be rotated through the positioning bracket to facilitate the collection of more and more comprehensive image data. It can be understood that the processor is electrically connected to the image acquisition device and the Internet of Vehicles communication device respectively.

[0039] In a specific embodiment, the position of the positioning unit is actually the longitude and latitude of the positioning unit, and may also be other data that can represent the position of the positioning unit, such as coordinates in a certain coordinate system.

[0040] In a specific embodiment, a positioning simulation image is obtained by real-time rendering based on the positioning information. The positioning simulation image is a two-dimensional or three-dimensional virtual image used to express the vehicle position and obstacle position in a limited image, providing a data basis for the vehicle to make early predictions and emergency avoidance.

[0041] In one embodiment, obtaining positioning information through an image acquisition device includes:

[0042] Acquiring a real-time image of the vehicle by an image acquisition device provided on the first vehicle;

[0043] The positioning unit extracts and analyzes the pre-processed real-time vehicle image through a machine learning algorithm and generates positioning information.

[0044] It is understandable that after obtaining the real-time image of the vehicle through the image acquisition device, the real-time image of the vehicle needs to be preprocessed by image segmentation, image denoising, grayscale conversion, etc., and the data including the obstacle position, the offset information of the obstacle position relative to the positioning unit position, and the offset information of the positioning unit position relative to the geometric center of the first vehicle are extracted from the preprocessed image, and finally integrated into the positioning information.

[0045] In an optional embodiment, the image acquisition device includes a camera, a driving recorder, and other vehicle-mounted devices that can acquire image data in real time and realize data interaction. Data acquisition and data interaction can be realized through the cooperation between the Internet of Vehicles and the vehicle-mounted devices.

[0046] In one embodiment, obtaining the offset information of the obstacle by the positioning unit includes:

[0047] Establish a three-dimensional rectangular coordinate system with the reference point of the image acquisition device as the origin, and mark it as the first coordinate system;

[0048] Extract obstacles from the real-time image of the vehicle through a machine learning algorithm, calculate the coordinates of the obstacles in the first coordinate system, and mark them as the first coordinates;

[0049] A three-dimensional rectangular coordinate system is established with the positioning unit position as the origin and marked as the second coordinate system. The coordinate system transformation method combines the first coordinates to obtain the coordinates of the obstacle in the second coordinate system and mark them as the second coordinates. The second coordinates are the offset information of the obstacle relative to the positioning unit position.

[0050] This embodiment obtains the offset information of the obstacle relative to the positioning unit based on the coordinate system conversion method, and further obtains the offset information of the obstacle relative to the geometric center of the vehicle, recording the obstacle information from multiple angles.

[0051] In an optional embodiment, in combination with the size and model of the vehicle, the relative position relationship between the vehicle and the obstacle can be obtained based on the offset information of the obstacle relative to the positioning unit or the geometric center of the vehicle, thereby avoiding collision between the vehicle and the obstacle and providing a data basis for the safe operation of the vehicle.

[0052] In one embodiment, the cloud platform analyzes and supplements the positioning information, including:

[0053] The cloud platform divides the positioning information according to the position of the first vehicle and generates five pieces of position information;

[0054] When the five position information are continuous and not empty, it is determined that the positioning information does not need to be supplemented; otherwise, the empty position information is supplemented by the auxiliary equipment.

[0055] In this embodiment, the five-position information is divided according to the five positions of the vehicle, and the five positions specifically include the front, rear, left, right and top of the vehicle; in other preferred embodiments, the bottom of the vehicle can also be taken into consideration to truly achieve all-round visual recognition.

[0056] It is worth noting that when the five orientation information are continuous and not empty, it is determined that the positioning information does not need to be supplemented; the continuity of the orientation information in this embodiment means that the images of the front, rear, left and right sides of the vehicle can be spliced ​​together to form a continuous image, that is, 360° without blind spots on the horizontal plane of the vehicle, and the orientation information in these four directions and the orientation information above can also be spliced ​​together continuously, that is, a hemispherical or ellipsoidal detection area with the geometric position of the vehicle as the center of the sphere is formed, and the image is continuous in this hemispherical or multi-spherical detection area.

[0057] It can be understood that the orientation information is not empty, that is, the part of the positioning information at this orientation is not empty, which actually means that no image data is collected at this orientation, resulting in the positioning information being empty, rather than that there is no obstacle information at this orientation.

[0058] In this embodiment, the position information of the empty space is supplemented by auxiliary equipment. The auxiliary equipment referred to in this embodiment generally refers to image acquisition equipment that is not related to the vehicle, such as surveillance cameras at intersections, surveillance cameras on both sides of the road, and image acquisition equipment installed on other vehicles. It is worth noting that when the positioning information is supplemented by the image data of the image acquisition equipment on other vehicles, it is necessary to obtain relevant data interaction permissions.

[0059] In one embodiment, the cloud platform sends the positioning simulation image to the target vehicle; wherein the target vehicle is a vehicle that has been started in the target area.

[0060] In this embodiment, the cloud platform sends the positioning simulation image to the target vehicle, and the target vehicle can obtain obstacle information in a timely manner through the positioning simulation image, and then determine its own position and safety.

[0061] In an optional embodiment, after the target vehicle receives the positioning information, it can combine the geometric center position of the first vehicle and the geometric center position of the target vehicle to obtain the distance of the obstacle relative to the target vehicle, providing a data basis for the safe driving of the target vehicle.

[0062] In a specific embodiment, acquiring the target area includes:

[0063] A circular area is drawn with the positioning reference point of the first vehicle as the center and the set length as the radius, and is marked as the target area; or

[0064] A rectangular area is drawn with the positioning reference point of the first vehicle as the center, the road width as the set width, and the set length, and marked as the target area.

[0065] It is understandable that the set length should be at least greater than or equal to the width of the first vehicle. Therefore, the size of the target area obtained for different vehicle models is different.

[0066] In one embodiment, when the positioning information is sent to the cloud platform, the positioning information is also sent to the Internet of Vehicles communication device of the adjacent vehicle. The adjacent vehicle determines the position of the first vehicle and the position of the obstacle based on the positioning information of the first vehicle.

[0067] In this embodiment, adjacent vehicles specifically refer to vehicles that are adjacent to the first vehicle and are within a set distance from the first vehicle, such as vehicles in front of, behind, on the left or on the right of the first vehicle; in this embodiment, the cloud platform can promptly make up for the shortcoming of slow speed in acquiring positioning simulation images, and provide data support for the safe driving of adjacent vehicles in a timely manner.

[0068] In a specific embodiment, the set distance may be a safe braking distance of the first vehicle.

[0069] Working principle of the present invention:

[0070] The positioning unit obtains the positioning information of the first vehicle in combination with the image acquisition device, and sends the positioning information to the cloud platform and adjacent vehicles respectively through the vehicle network communication device.

[0071] The cloud platform judges and supplements the positioning information, generates a positioning simulation image based on the positioning information, and sends the positioning simulation image to the Internet of Vehicles communication equipment of other vehicles.

[0072] When the positioning information is sent to the cloud platform, the positioning information is also sent to the vehicle network communication equipment of the adjacent vehicle. The adjacent vehicle determines the position of the first vehicle and the obstacle position based on the positioning information of the first vehicle.

[0073] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A deep learning vision-assisted positioning method for Internet of Vehicles, characterized by: include: The positioning unit acquires positioning information of the first vehicle in conjunction with the image acquisition device, and transmits the positioning information to the cloud platform and adjacent vehicles via the vehicle network communication device; wherein the positioning information includes offset information of the obstacle relative to the positioning unit position, the positioning unit position, and offset information of the positioning unit position relative to the positioning reference point of the first vehicle; The cloud platform judges and supplements the positioning information, generates a positioning simulation image based on the positioning information, and sends the positioning simulation image to the Internet of Vehicles communication equipment of other vehicles; the positioning simulation image is rendered and obtained in real time based on the positioning information; Obtaining the offset information of the obstacle by the positioning unit includes: Establish a three-dimensional rectangular coordinate system with the reference point of the image acquisition device as the origin, and mark it as the first coordinate system; Extract obstacles from the real-time image of the vehicle through a machine learning algorithm, calculate the coordinates of the obstacles in the first coordinate system, and mark them as the first coordinates; A three-dimensional rectangular coordinate system is established with the positioning unit position as the origin, and is marked as the second coordinate system. The coordinate system transformation method combines the first coordinates to obtain the coordinates of the obstacle in the second coordinate system, and is marked as the second coordinates. The second coordinates are the offset information of the obstacle relative to the positioning unit position. The cloud platform judges and supplements the positioning information, including: The cloud platform divides the positioning information according to the position of the first vehicle and generates five pieces of position information; When the five position information are continuous and not empty, it is determined that the positioning information does not need to be supplemented; otherwise, the empty position information is supplemented by the auxiliary equipment.

2. The method for deep learning visual assisted positioning of Internet of Vehicles according to claim 1, characterized in that: Acquiring positioning information through the image acquisition device includes: Acquire a real-time image of the vehicle through an image acquisition device provided on the first vehicle; wherein the image acquisition device includes a camera and a driving recorder; The positioning unit extracts and analyzes the pre-processed real-time vehicle image through a machine learning algorithm and generates positioning information; the pre-processing includes image segmentation, image denoising and grayscale transformation.

3. The method for deep learning visual assisted positioning of Internet of Vehicles according to claim 2, characterized in that: The positioning reference point is the geometric center of the first vehicle.

4. The method for deep learning visual assisted positioning of Internet of Vehicles according to claim 1, characterized in that: The positioning unit includes a positioning bracket and a processor; wherein the positioning bracket is used to install the image acquisition device, and the processor communicates and / or is electrically connected with the image acquisition device and the vehicle network communication device respectively.

5. The method for deep learning visual assisted positioning of Internet of Vehicles according to claim 1, characterized in that: The cloud platform sends the positioning simulation image to the target vehicle; wherein the target vehicle is a vehicle that has been started in the target area, and the target area is drawn based on the positioning reference point of the first vehicle.

6. The method for deep learning visual assisted positioning of Internet of Vehicles according to claim 5, characterized in that: The acquisition of the target area includes: A circular area is drawn with the positioning reference point of the first vehicle as the center and the set length as the radius, and is marked as the target area; or A rectangular area is drawn with the positioning reference point of the first vehicle as the center, the road width as the set width, and the set length, and marked as the target area.

7. The method for deep learning visual assisted positioning of an Internet of Vehicles according to claim 1, characterized in that: When the positioning information is sent to the cloud platform, the positioning information is also sent to the vehicle network communication equipment of the adjacent vehicle. The adjacent vehicle determines the position of the first vehicle and the obstacle position based on the positioning information of the first vehicle.

Images