Method and system for analyzing and reminding road condition of familiar road based on panoramic image, and medium

Abstract

The invention relates to a panoramic image-based familiar road condition analysis reminding method and system and a medium. The method comprises the following steps: 1, generating road condition information; step 2, processing cloud data; 3, prompting the road condition of the vehicle; the invention aims to fill up the current situation that the existing navigation application cannot remind the state of a road, and designs a familiar road condition reminding device, a terminal and a storage medium which are used for recording and analyzing the road condition of a driving road based on a vehicle-mounted panoramic image and can remind a driver of the current vehicle condition in real time. According to the invention, missing points of existing driving condition reminding can be supplemented, the driver is reminded of the road condition ahead and matters needing attention in real time, and the driving safety and the driving experience are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle networking, and in particular to a method, system and medium for analyzing and reminding familiar road conditions based on panoramic images. Background Art

[0002] The traffic reminders of existing in-vehicle navigation applications can display the degree of traffic congestion and smoothness in real time in the navigation information, so that drivers can better choose driving routes and avoid congestion and accidents. However, there is no way to remind drivers to pay attention to potholes based on the state of the road itself (road flatness, road surface wear, etc.), and switch driving modes according to different road conditions. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the above-mentioned problems existing in the prior art, and to provide a method, system and medium for analyzing and reminding familiar road conditions based on panoramic images.

[0004] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0006] A method for analyzing and reminding familiar road conditions based on panoramic images comprises the following steps:

[0007] Step 1: Generate traffic information;

[0008] Step 2: Process cloud data;

[0009] Step 3: Prompt vehicle road conditions.

[0010] Furthermore, the generation of road condition information described in step 1 specifically includes: during the driving of the vehicle, the vehicle driving road condition is recorded in real time through the on-board panoramic imaging lens, the camera inputs the video stream to the image stitching processor, the image stitching processor optimizes, dedistorts, crops and stitches the video, and transmits the processed and stitched video stream to the video road condition analysis processor, the road condition analysis processor performs noise reduction, segmentation and stitching on the input video stream, converts the video into road condition pictures at a certain time interval, and then performs image analysis on the road surface picture to determine whether there are cracks, potholes, and whether the road surface is smooth or rough, and generates a final analysis result by corresponding the processed result of each road surface picture to the segmentation time point one by one, and stores the analysis result in the local database; at the same time, during the driving process, the current positioning information is notified in real time through the positioning device, and stored in the local database together, and associated with the road condition information according to time; when the local data accumulates to a certain amount, this segment of data is packaged and uploaded to the cloud for storage; the cloud database integrates the vehicle condition data from different sources.

[0011] Furthermore, the processing of cloud data in step 2 specifically includes: sorting and processing the data uploaded by each local terminal, and merging and storing the data in the same urban area.

[0012] Furthermore, the prompting of vehicle road conditions in step three specifically includes: when the vehicle is traveling on the road, analyzing the vehicle's position and driving direction based on positioning, and obtaining the next road condition in the direction of movement from a database, analyzing the road condition, and determining whether there is a significant change in the subsequent road condition compared to the current one; if there is a change, reminding the user of the change in road condition and suggesting switching driving modes.

[0013] Furthermore, cloud computing replaces the video processor and image analyzer in step one, and directly uploads the original video stream and corresponding positioning information to the cloud. The cloud performs video processing and image analysis, and directly stores the analyzed information in the cloud database, thereby reducing performance loss on the vehicle side. The vehicle side is required to cache and package the original video data and corresponding positioning data at intervals, and upload them to the cloud for processing when the network connection is normal.

[0014] A familiar road condition analysis and reminder system based on panoramic images, comprising:

[0015] The road condition information generation module is used to collect the road video and location information of the vehicle and generate the road condition data with timestamp and longitude and latitude;

[0016] The cloud data processing module is used to sort and process the data uploaded by each local terminal, and merge and store the data in the same city area;

[0017] The vehicle road condition prompt module is used to prompt the user in real time about changes in road conditions and driving precautions when the vehicle is driving on a road with recorded road condition information.

[0018] Furthermore, a familiar road condition analysis and reminder system based on panoramic images also includes a cloud computing module, which is used to use cloud computing capabilities to upload video files directly to the cloud for analysis to improve the accuracy of the analysis.

[0019] A device, characterized in that: it includes one or more processors;

[0020] A memory for storing one or more programs;

[0021] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0022] A computer-readable storage medium stores a computer program, which implements the method described above when executed by a processor.

[0023] A vehicle adopts the familiar road condition analysis and reminder system based on panoramic images as described above.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention aims to fill the current situation that existing navigation applications cannot remind the status of the road itself. It designs a road condition reminder device, terminal and storage medium that records and analyzes the road conditions of the roads traveled based on the on-board panoramic image, and can remind the driver of the current vehicle condition in real time on familiar roads (not the first time driving or roads with road condition records). It can make up for the shortcomings of existing driving condition reminders, remind the driver of the road conditions ahead, precautions, etc. in real time, and can effectively improve driving safety and driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention will be further described below in conjunction with the accompanying drawings:

[0027] Figure 1 A schematic diagram of the vehicle driving conditions and data processing flow for the on-board panoramic image recording;

[0028] Figure 2 A schematic diagram of converting the input video into segmented road condition data;

[0029] Figure 3 A schematic diagram for sorting and processing the data uploaded by each local terminal and merging and storing the data;

[0030] Figure 4 Provide a flow chart for vehicle road condition prompts;

[0031] Figure 5 This is a flow chart of the familiar road condition analysis and reminder method based on panoramic images according to the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the implementation of the present invention clearer, the technical scheme in the embodiment of the present invention will be described in more detail below in conjunction with the drawings in the embodiment of the present invention. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of the present invention, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limitations on the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0033] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the scope of protection of the present invention.

[0034] The present invention is described in detail below in conjunction with the accompanying drawings:

[0035] The present invention provides an embodiment, see Figure 5 A method for analyzing and reminding familiar roads based on panoramic images mainly implements real-time road condition reminders through the following steps:

[0036] 1. Generation of traffic information

[0037] See also Figure 1During the driving process of the vehicle, the vehicle driving road conditions are recorded in real time through the on-board panoramic image lens. Taking the common on-board panoramic image as an example, the common on-board panoramic image generally has four ultra-wide-angle lenses in front, behind, left and right. The four-way camera will input the video stream to the image stitching processor, which will optimize, dedistort, crop and stitch the video. The processed and stitched video stream is transmitted to the video road condition analysis processor. The road condition analysis processor will perform noise reduction, segmentation and splicing on the input video stream, and convert the video into road condition pictures at a certain time interval. Then, the road condition pictures are analyzed to determine whether there are cracks, potholes, and whether the road surface is smooth or rough. The processed results of each road surface picture are matched with the segmentation time point to generate the final analysis result, and the analysis result is stored in the local database; at the same time, during the driving process, the current positioning information will be notified in real time through the positioning device, and stored in the local database together, and associated with the road condition information according to time. When the local data accumulates to a certain amount, this segment of data is packaged and uploaded to the cloud for storage. The cloud database can integrate vehicle condition data from different sources, enrich the road condition database, fill the singleness of the local database, and pull cloud road condition data for reminders when the vehicle is connected to the Internet.

[0038] See also Figure 2 , which is a brief description of the steps for traffic video analysis, from converting the input video into segmented traffic data.

[0039] Cloud computing is an optional optimization solution. Cloud computing replaces or partially replaces the above-mentioned video processor and image analyzer and can be executed in the cloud. The original video stream and corresponding positioning information are directly uploaded to the cloud, which performs video processing and image analysis. After analysis, it is directly stored in the cloud database, thereby reducing the performance loss of the vehicle side (the configuration and performance of the vehicle side are limited, while the resources and capabilities that can be used by the cloud are more sufficient). It should be noted that the alternative solution is highly dependent on the network and has poor real-time processing capabilities, so the vehicle side needs to cache and package the original video data and corresponding positioning data for a period of time (in order to reduce the frequency of network requests and reduce unnecessary performance consumption), and upload them to the cloud for processing when the network connection is normal. The vehicle side packages the original video files and positioning information for a period of time, and uploads the file package through the cloud interface when the network is connected. The cloud performs video processing and image analysis, and stores the processing results directly in the cloud database.

[0040] Using cloud computing power to upload video files directly to the cloud for analysis can improve the accuracy of the analysis, but the disadvantage is that it depends on the network. It can be used as an alternative and optimization direction.

[0041] 2. Cloud Data Processing

[0042] See also Figure 3, sort and process the data uploaded by each local terminal, and merge and store the data in the same urban area.

[0043] 3. Vehicle and road condition reminder

[0044] When the vehicle is driving on the road, the location and driving direction of the vehicle are analyzed based on the positioning, and the road conditions in the direction of movement are obtained from the database. The road conditions are analyzed to determine whether the subsequent road conditions have changed significantly compared to the current ones. If there are changes, the user is reminded of the change in road conditions and is advised to switch driving modes. The flowchart is as follows Figure 4 .

[0045] General road condition reminders are based on traffic conditions such as vehicle congestion and accidents, and do not remind drivers to drive carefully based on the road conditions themselves. The focus of the present invention is to collect, analyze and store data on the road conditions themselves, and to enrich and share data through the network, so as to remind drivers to pay attention to changes in vehicle conditions during driving.

[0046] The present invention provides another embodiment, a familiar road condition analysis and reminder system based on panoramic images, comprising:

[0047] The road condition information generation module is used to collect the road video and location information of the vehicle and generate the road condition data with timestamp and longitude and latitude;

[0048] The cloud data processing module is used to sort and process the data uploaded by each local terminal, and merge and store the data in the same city area;

[0049] The vehicle road condition prompt module is used to remind the user of road condition changes and driving precautions in real time when the vehicle is driving on a road with recorded road condition information;

[0050] The cloud computing module is used to utilize cloud computing capabilities to upload video files directly to the cloud for analysis to improve the accuracy of the analysis.

[0051] Based on the above-mentioned familiar road condition analysis and reminder method based on panoramic images, the present invention provides another device. The device includes but is not limited to: one or more processors and memories.

[0052] The memory, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions corresponding to the familiar road condition analysis and reminder method based on panoramic images in the embodiment of the present invention. The processor executes various functional applications and data processing of the vehicle by running the software programs, instructions and modules stored in the memory, that is, realizing the familiar road condition analysis and reminder method based on panoramic images.

[0053] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0054] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a method for analyzing and reminding a familiar road condition based on panoramic images is implemented. The method for analyzing and reminding a familiar road condition based on panoramic images comprises the following steps:

[0055] Step 1: Generate traffic information;

[0056] Step 2: Process cloud data;

[0057] Step 3: Prompt vehicle road conditions.

[0058] The computer-readable storage medium provided by the present invention has computer executable instructions that are not limited to the method operations described above, but can also execute related operations in the familiar road condition analysis and reminder method based on panoramic images provided by any embodiment of the present invention.

[0059] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0060] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (for example, coaxial cable, optical fiber, digital subscriber line DSL) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk Solid State Disk, SSD), etc.

[0061] In the above embodiments, the various units and modules included are divided only according to functional logic, but are not limited to the above divisions, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0062] The present invention provides another embodiment, a vehicle, which uses the above-mentioned familiar road condition analysis and reminder system based on panoramic images, and the system includes:

[0063] The road condition information generation module is used to collect the road video and location information of the vehicle and generate the road condition data with timestamp and longitude and latitude;

[0064] The cloud data processing module is used to sort and process the data uploaded by each local terminal, and merge and store the data in the same city area;

[0065] The vehicle road condition prompt module is used to remind the user of road condition changes and driving precautions in real time when the vehicle is driving on a road with recorded road condition information;

[0066] The cloud computing module is used to utilize cloud computing capabilities to upload video files directly to the cloud for analysis to improve the accuracy of the analysis.

[0067] The above is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent replacement and improvement made by any person skilled in the art within the technical scope disclosed by the present invention and within the spirit and principle of the present invention shall be covered within the protection scope of the present invention. At the same time, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A method for analyzing and reminding familiar road conditions based on panoramic images, characterized in that: The following steps are involved: Step 1: Generate traffic information; Step 2: Process cloud data; Step 3: Prompt vehicle road conditions.

2. The method for familiar road condition analysis and reminder based on panoramic images according to claim 1, characterized in that: The step of generating road condition information in step 1 specifically includes: during the driving process of the vehicle, the vehicle driving road condition is recorded in real time through the on-board panoramic imaging lens, the camera inputs the video stream to the image stitching processor, the image stitching processor optimizes, dedistorts, crops and stitches the video, and transmits the processed and stitched video stream to the video road condition analysis processor, the road condition analysis processor performs noise reduction processing, segmentation and stitching on the input video stream, converts the video into road condition pictures at a certain time interval, and then performs image analysis on the road surface pictures to determine whether there are cracks, potholes, and whether the road surface is smooth or rough, and generates a final analysis result by corresponding the processed result of each road surface picture to the segmentation time point one by one, and stores the analysis result in the local database; at the same time, during the driving process, the current positioning information is notified in real time through the positioning device, and stored in the local database together, and associated with the road condition information according to time; when the local data accumulates to a certain amount, this segment of data is packaged and uploaded to the cloud for storage; the cloud database integrates the vehicle condition data from different sources.

3. The method for familiar road condition analysis and reminder based on panoramic images according to claim 1, characterized in that: The processing of cloud data in step 2 specifically includes: sorting and processing the data uploaded by each local terminal, and merging and storing the data in the same urban area.

4. The method for analyzing and reminding familiar road conditions based on panoramic images according to claim 1, characterized in that: The prompting of vehicle road conditions described in step three specifically includes: when the vehicle is traveling on the road, analyzing the vehicle's position and driving direction based on positioning, and obtaining the next road condition in the direction of movement from the database, analyzing the road condition, and determining whether there is a significant change in the subsequent road condition compared to the current one. If there is a change, the user is reminded of the change in road conditions and is advised to switch driving modes.

5. The method for familiar road condition analysis and reminder based on panoramic images according to claim 2, characterized in that: Cloud computing replaces the video processor and image analyzer in step one, and directly uploads the original video stream and corresponding positioning information to the cloud. The cloud performs video processing and image analysis, and directly stores the analyzed data into the cloud database, thereby reducing the performance loss on the vehicle side. The vehicle side is required to cache and package the original video data and corresponding positioning data at intervals, and upload them to the cloud for processing when the network connection is normal.

6. A familiar road condition analysis and reminder system based on panoramic images, characterized in that: include: The road condition information generation module is used to collect the road video and location information of the vehicle and generate the road condition data with timestamp and longitude and latitude; The cloud data processing module is used to sort and process the data uploaded by each local terminal, and merge and store the data in the same city area; The vehicle road condition prompt module is used to prompt the user in real time about changes in road conditions and driving precautions when the vehicle is driving on a road with recorded road condition information.

7. The familiar road condition analysis and reminder system based on panoramic images according to claim 6, characterized in that: It also includes a cloud computing module for utilizing cloud computing capabilities to upload video files directly to the cloud for analysis to improve the accuracy of the analysis.

8. A device, characterized in that: comprising one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

10. A vehicle, characterized in that: A familiar road condition analysis and reminder system based on panoramic images as described in claim 6 or claim 7 is adopted.

Images