Intelligent Management Method and System for Foreign Objects on Highway Pavement Based on Image Analysis

Abstract

The present application discloses an intelligent management method and system for foreign objects on highway pavements based on image analysis, belonging to the field of image analysis. The present application conducts road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types, conducts vehicle driving danger prediction on the road based on the running conditions of the vehicle and the parameter comparison between the vehicle and the foreign object, conducts driving abnormality prediction of the vehicle based on the obtained road abnormal danger analysis results and the vehicle driving danger prediction results on the road, and conducts comprehensive analysis based on the physical characteristics and distribution of the predicted foreign object types, combined with the running state of the vehicle and the parameter comparison between the vehicle and the foreign object, which can more comprehensively and accurately predict the driving abnormality of the vehicle, so as to significantly improve road safety.

Description

Technical Field

[0001] This application belongs to the field of image analysis, specifically an intelligent management method and system for foreign objects on highway pavements based on image analysis. Background Art

[0002] Road traffic safety is one of the key concerns in modern society, and road abnormal danger analysis and vehicle driving danger prediction are the key technologies to ensure road traffic safety. With the acceleration of logistics transportation and urbanization, the number of transport vehicles and foreign objects on the road is increasing continuously. How to accurately identify and predict road abnormal dangers has become an important issue in improving road traffic safety. In the existing technology, road abnormal danger analysis and cargo identification mainly rely on the similarity analysis between transported goods and foreign objects. This method judges the type of foreign objects by comparing the appearance features of goods and foreign objects, such as color and size. However, this method has obvious limitations. For example, when the goods transported by two vehicles within a certain time period are highly similar in color and size, such as iron balls and ball models, it is difficult for the existing technology to accurately distinguish them, resulting in identification errors and unable to effectively predict potential dangers.

[0003] In addition, single similarity analysis cannot comprehensively reflect the impact of the physical properties of foreign objects on driving safety, such as the hardness, elasticity, and density of foreign objects. These physical properties are directly related to the damage degree and rebound possibility of foreign objects when hit by vehicles. Therefore, the existing technology has the problem of insufficient prediction accuracy in practical applications and cannot meet the requirements of intelligent driving and logistics transportation for high-precision danger prediction.

[0004] In view of this, this application designs an intelligent management method and system for foreign objects on highway pavements based on image analysis. Summary of the Invention

[0005] To solve the deficiencies in the existing technology mentioned in the background art, this application proposes an intelligent management method and system for foreign objects on highway pavements based on image analysis. This application conducts road abnormal danger analysis based on the physical properties and distribution of the predicted foreign object types, conducts road vehicle driving danger prediction based on the running conditions of the vehicle and the parameter comparison between the vehicle and the foreign object, conducts vehicle driving abnormality prediction based on the obtained road abnormal danger analysis results and road vehicle driving danger prediction results, and conducts comprehensive analysis based on the physical properties and distribution of the predicted foreign object types, combined with the running state of the vehicle and the parameter comparison between the vehicle and the foreign object, which can more comprehensively and accurately predict vehicle driving abnormalities and significantly improve road safety.

[0006] To achieve the above object, this application provides the following technical solutions: In the first aspect, this application provides an intelligent management method for foreign objects on highway pavements based on image analysis, which includes the following specific steps:

[0007] Step 1: Obtain the image information of foreign objects on the road surface of the highway, and at the same time obtain the data of passing vehicles on the road for the analysis of the types of foreign objects;

[0008] Step 2: Conduct an analysis of the abnormal danger of the road based on the physical properties and distribution of the types of foreign objects predicted;

[0009] Step 3: Predict the driving danger of vehicles on the road based on the running conditions of the vehicles and the comparison of the parameters between the vehicles and the foreign objects;

[0010] Step 4: Predict the abnormal driving of vehicles based on the obtained results of the analysis of the abnormal danger of the road and the results of the prediction of the driving danger of vehicles on the road;

[0011] Step 5: Remind of danger signals based on the results of the prediction of abnormal vehicle driving.

[0012] Preferably, based on the above solution, the specific content of obtaining the image information of foreign objects on the road surface of the highway and at the same time obtaining the data of passing vehicles on the road for the analysis of the types of foreign objects is as follows:

[0013] Step 101: Obtain the image information of foreign objects on the road surface of the highway through an image acquisition terminal, including the shape information and color information of the corresponding image. At the same time, obtain the information of external attachments and transported goods reported by the vehicles passing by the position of the road foreign objects at each moment. Among them, the information of transported goods and external attachments includes the type of goods, the shape information and color information of the goods, and also includes the material property data of the goods such as hardness, elasticity, and density; at the same time, obtain the passing duration and passing distance of the passing vehicles;

[0014] Step 102: Conduct an analysis of the similarity of items based on the image information of foreign objects on the road surface of the highway and the information of transported goods and external attachments of the vehicles. The method of analyzing the similarity of items is as follows:

[0015] Obtain the shape information of the corresponding foreign object image and the color information of each point to construct the image feature vector of the foreign object image. Obtain the shape information and color information of the transported goods and external attachments of each vehicle to construct the image feature vector of the transported goods and external attachments image. Import the image feature vector of the foreign object image and the image feature vectors of the transported goods and external attachments images into the cosine similarity calculation formula to calculate the cosine similarity between the foreign object image and each transported item of each vehicle. Among them, the cosine similarity calculation formula between the foreign object image and the i-th transported item of the d-th vehicle is: , where, is the image feature vector of the foreign object image, is the feature vector of the i-th transported item of the d-th vehicle, is the modulus of the vector;

[0016] Step 103: Obtain the transported goods of the corresponding vehicles with a cosine similarity greater than or equal to the set similarity threshold, obtain the distances and durations from the driving positions of these vehicles to the appearance of the foreign object, and calculate the selection value based on the distances and durations from the driving positions of the vehicles to the appearance of the foreign object and the cosine similarity between the foreign object image and the transported goods of the vehicles. Among them, the formula for calculating the selection value of the i-th transported good of the d-th vehicle is: , where Lm is the set standard distance, Ldi is the minimum distance from the driving position of the vehicle to the appearance of the foreign object, tm is the set standard duration, tdi is the duration from when the vehicle travels to the corresponding driving position to the appearance of the foreign object, and set the type of transported good corresponding to the maximum selection value as the type of foreign object;

[0017] Preferably, based on the above solution, the analysis of road abnormal danger based on the physical properties and distribution of the predicted foreign object types includes the following specific steps:

[0018] Step 201: Obtain the hardness, elasticity, and density of the items corresponding to the foreign object types, and at the same time obtain the volume of the foreign object and its distribution on the road;

[0019] Step 202: Analyze the road danger situation based on the hardness, elasticity, and density of the items corresponding to the foreign object types, the volume of the foreign object, and its distribution on the road. Among them, the formula for analyzing road abnormal danger is: , where M is the number of data types of the hardness, elasticity, and density of the items, gi is the influence weight of the i-th data type in the data types of the hardness, elasticity, and density of the items on the impact danger. By collecting the danger degree data of different foreign objects under different conditions through experiments, according to the experimental data, adjust the weight coefficient through machine learning algorithms (such as regression analysis or neural network), vi is the data value of the i-th data type in the i-th data type of the hardness, elasticity, and density of the items, vim is the standard value of the i-th data type in the data types of the hardness, elasticity, and density of the items, Vs is the sum of the volumes of all foreign objects, ls is the average distance between foreign objects, and Rs is the road surface width.

[0020] Preferably, based on the above solution, the prediction of vehicle driving danger on the road based on the operating conditions of the vehicle and the parameter comparison between the vehicle and the foreign object includes the following specific steps:

[0021] Step 301: Obtain the driving operating conditions of the vehicle, and analyze the vehicle driving danger based on the driving operating conditions of the vehicle. Among them, the formula for analyzing vehicle driving danger is: , where T is the driving time of the previous cycle, ht is the left and right swing amplitude of the vehicle during driving, and hm is the left and right swing safety amplitude of the vehicle during driving;

[0022] Step 302: Obtain the driving dimension data of the vehicle and the position data of the foreign object on the road, and perform driving anomaly analysis based on the driving dimension data of the vehicle and the position data of the foreign object on the road. The driving anomaly analysis formula is as follows: , where Cr is the width of the vehicle, Crm is the average distance between foreign objects, Lrm is the height of the foreign object, and Lr is the chassis height of the vehicle; the vehicle width is an important parameter of the vehicle size, which directly affects the stability and passing ability of the vehicle; the distribution density of objects directly affects the safety of the vehicle during driving, and the greater the density, the higher the degree of danger; the foreign object height determines the possibility of contact between the vehicle chassis and the foreign object; the chassis height is an important parameter in vehicle design, which directly affects the safety of the vehicle when passing foreign objects. In order to normalize the parameters, the formulas and are used;

[0023] Step 303: Perform weighted summation on the obtained vehicle driving risk analysis result and driving anomaly analysis result to obtain the vehicle driving risk prediction result.

[0024] Preferably, based on the above solution, the driving anomaly prediction of the vehicle based on the obtained road anomaly risk analysis result and the vehicle driving risk prediction result of the road includes the following specific contents:

[0025] Obtain the calculated vehicle driving risk prediction result and road anomaly risk analysis result, and multiply them to obtain the vehicle driving anomaly prediction result;

[0026] Set the driving anomaly threshold. If the driving anomaly prediction result is greater than or equal to the driving anomaly threshold, it indicates that the vehicle will be in danger during driving, and the vehicle is reminded to change the road as soon as possible;

[0027] If the driving anomaly prediction result is less than the driving anomaly threshold and greater than or equal to 70% of the driving anomaly threshold, the driver is reminded to pay attention to the road conditions;

[0028] If the driving anomaly prediction result is less than 70% of the driving anomaly threshold, it indicates that the driving is safe.

[0029] Preferably, based on the above solution, the reminder of the danger signal based on the vehicle driving anomaly prediction result includes the following specific steps: Obtain the vehicle information driving to this section in the next stage, and send the corresponding signal of driving in danger, reminding the driver to pay attention to the road conditions or driving safely to the vehicle driver through the corresponding communication method, and at the same time, send the position and type information of the foreign object to the road management department in real time to remind the road management department to clean it up.

[0030] In a second aspect, the present application provides an intelligent management system for foreign objects on a highway pavement based on image analysis, which is implemented based on the above-mentioned intelligent management method for foreign objects on a highway pavement based on image analysis. Specifically, it includes a foreign object type analysis module, a road abnormal danger analysis module, a vehicle driving danger prediction module, a driving abnormality prediction module, and a signal reminder module. Among them, the foreign object type analysis module obtains the image information of foreign objects on the highway pavement and simultaneously obtains the data of vehicles passing through the road for the analysis of foreign object types. The road abnormal danger analysis module conducts road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types. The vehicle driving danger prediction module predicts the vehicle driving danger on the road based on the running conditions of the vehicle and the comparison of parameters between the vehicle and the foreign object. The driving abnormality prediction module predicts the driving abnormality of the vehicle based on the obtained road abnormal danger analysis results and the vehicle driving danger prediction results on the road. The signal reminder module gives a reminder of a danger signal based on the vehicle driving abnormality prediction results.

[0031] In a third aspect, the present application provides an electronic device, including: a processor and a memory. Among them, a computer program that can be called by the processor is stored in the memory;

[0032] The processor executes the above-mentioned intelligent management method for foreign objects on a highway pavement based on image analysis by calling the computer program stored in the memory.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium storing instructions, which, when run on a computer, cause the computer to execute the intelligent management method for foreign objects on a highway pavement based on image analysis as described above.

[0034] Meanwhile, compared with the prior art, the technical effects and advantages of the present application are:

[0035] The first advantage of the present application is that the present application obtains the image information of foreign objects on the highway pavement and simultaneously obtains the data of vehicles passing through the road for the analysis of foreign object types, which greatly improves the recognition accuracy of foreign objects and lays a foundation for subsequent driving abnormality analysis.

[0036] The second advantage of the present application is that the present application conducts road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types, predicts the vehicle driving danger on the road based on the running conditions of the vehicle and the comparison of parameters between the vehicle and the foreign object, predicts the driving abnormality of the vehicle based on the obtained road abnormal danger analysis results and the vehicle driving danger prediction results on the road, and conducts a comprehensive analysis based on the physical characteristics and distribution of the predicted foreign object types, combined with the running state of the vehicle and the comparison of parameters between the vehicle and the foreign object, which can more comprehensively and accurately predict the driving abnormality of the vehicle and significantly improve road safety. Description of the Drawings

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings;

[0038] Figure 1 It is a schematic diagram of the overall process of the intelligent management method for foreign objects on highway pavement based on image analysis;

[0039] Figure 2 It is a schematic diagram of the specific process of step 1 of the intelligent management method for foreign objects on highway pavement based on image analysis;

[0040] Figure 3 It is a schematic diagram of the specific process of step 3 of the intelligent management method for foreign objects on highway pavement based on image analysis;

[0041] Figure 4 It is a schematic diagram of the module composition of the intelligent management system for foreign objects on highway pavement based on image analysis. Detailed implementation manners

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. The description of at least one exemplary embodiment is actually only illustrative and in no way limits the present application and its application or use.

[0043] In addition, the accompanying drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions of them will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0044] It should be understood that although the terms "first", "second", etc. may be used here to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit can be called the second unit, and similarly the second unit can be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed related items.

[0045] To solve the technical problems raised in the background art, the present application provides a preferred embodiment: As Figures 1 - 3 shown, an intelligent management method for foreign objects on a highway pavement based on image analysis, which includes the following specific steps:

[0046] Step 1: Obtain the image information of foreign objects on the highway pavement, and at the same time obtain the vehicle data passing through the road for the analysis of the types of foreign objects;

[0047] Step 2: Conduct an analysis of the abnormal danger of the road based on the physical characteristics and distribution of the predicted types of foreign objects;

[0048] Step 3: Predict the driving danger of vehicles on the road based on the running conditions of the vehicles and the comparison of the parameters between the vehicles and the foreign objects;

[0049] Step 4: Predict the abnormal driving of vehicles based on the obtained results of the analysis of the abnormal danger of the road and the prediction results of the driving danger of vehicles on the road;

[0050] Step 5: Remind of danger signals based on the prediction results of abnormal vehicle driving;

[0051] In this embodiment, the specific content of obtaining the image information of foreign objects on the highway pavement and at the same time obtaining the vehicle data passing through the road for the analysis of the types of foreign objects is as follows:

[0052] Step 101: Obtain the image information of foreign objects on the highway pavement through an image acquisition terminal, including the shape information and color information of the corresponding image. At the same time, obtain the information of external attachments and transported goods reported by the vehicles passing through the position of the road foreign objects at each moment. Among them, the information of transported goods and external attachments includes the type of goods, the shape information and color information of the goods, and also includes the material property data of the goods such as hardness, elasticity and density; at the same time, obtain the passing duration and passing distance of the passing vehicles;

[0053] Foreign objects on the road are usually the transported goods and additional attachments that fall from vehicles during road transportation. Due to the obstruction of the vehicle or the influence of the image acquisition terminal itself, the specific process of the item falling is not captured. Thus, the type of the item is in doubt, and its impact on the road cannot be analyzed. Therefore, it is necessary to estimate the type of the item. Hence, it is required to obtain the information of the additional attachments and transported goods reported by the vehicles traveling from the position of the road foreign object at each moment through the vehicle management department terminal, and jointly analyze the probability of the type of the foreign object by using the duration and distance of the vehicle traveling to the appearance of the foreign object, as well as the similarity between the transported goods and the foreign object. However, the existing technology usually only uses the similarity between the transported goods and the foreign object to analyze the type of the foreign object. In this way, if similar transported goods are carried by two vehicles within a certain period of time, such as iron balls and rubber ball models that are similar in color and size, it will be difficult to distinguish them during identification. In contrast, this application comprehensively uses the duration and distance of the vehicle traveling to the appearance of the foreign object. That is, the shorter the distance and duration from the vehicle's traveling position to the appearance of the foreign object, the closer the vehicle is to the foreign object, which conversely indicates a high probability that the foreign object has fallen from the vehicle.

[0054] Step 102: Conduct item similarity analysis based on the image information of the foreign object on the road surface, the transported goods of the vehicle, and the additional attachment information. The item similarity analysis method is as follows:

[0055] Obtain the shape information of the corresponding foreign object image and the color information of each point to construct the image feature vector of the foreign object image. Obtain the item shape information and item color information of the transported goods and additional attachment information of each vehicle to construct the image feature vectors of the transported goods and additional attachment images. Import the image feature vector of the foreign object image and the image feature vectors of the transported goods and additional attachment images into the cosine similarity calculation formula to calculate the cosine similarity between the foreign object image and each transported item of each vehicle. Among them, the cosine similarity calculation formula between the foreign object image and the i-th transported item of the d-th vehicle is: , where is the image feature vector of the foreign object image, is the feature vector of the i-th transported item of the d-th vehicle, is the modulus of the vector;

[0056] Step 103: Obtain the transported items of the corresponding vehicles whose cosine similarity is greater than or equal to the set similarity threshold. Obtain the distance and duration from the traveling position of these vehicles to the appearance of the foreign object, and calculate the selection value based on the distance and duration from the traveling position of the vehicle to the appearance of the foreign object and the cosine similarity between the foreign object image and the transported items of the vehicle. Among them, the selection value calculation formula for the i-th transported item of the d-th vehicle is: , where Lm is the set standard distance, Ldi is the minimum distance from the driving position of the vehicle to the appearance of the foreign object, tm is the set standard duration, tdi is the duration from when the vehicle travels to the corresponding driving position to the appearance of the foreign object, and the type of the transported object corresponding to the maximum selected value is set as the type of the foreign object;

[0057] Exemplarily, the similarity threshold here is set to 90%. The standard distance and the standard duration here play a dual role in eliminating units and weights. The values of the standard distance and the standard duration can be adjusted to adjust the influence weights of the distance and the duration, which are obtained according to specific experiments. If the cosine similarity between the foreign object image and the transported objects of each vehicle is less than the similarity threshold, all are substituted into the selection value calculation formula to calculate the selection values of the transported objects of each vehicle. If there are multiple foreign objects of the same type, Ldi takes the minimum distance from the driving position of the vehicle to the appearance of all foreign objects, and tdi takes the minimum duration from when the vehicle travels to the corresponding driving position to the appearance of one of the foreign objects;

[0058] In this embodiment, the analysis of the abnormal road danger based on the physical characteristics and distribution of the predicted foreign object types includes the following specific steps:

[0059] Step 201: Obtain the hardness, elasticity, and density of the item corresponding to the foreign object type, and at the same time obtain the volume of the foreign object and its distribution on the road;

[0060] Step 202: Analyze the road danger situation based on the hardness, elasticity, and density of the item corresponding to the foreign object type, the volume of the foreign object, and its distribution on the road. Among them, the road abnormal danger analysis formula is: , where M is the number of data types of the hardness, elasticity, and density of the item, gi is the influence weight of the i-th data type in the data types of the hardness, elasticity, and density of the item on the impact danger. By collecting the danger degree data of different foreign objects under different conditions through experiments, according to the experimental data, the weight coefficient is adjusted through machine learning algorithms (such as regression analysis or neural network), vi is the data value of the i-th data type in the i-th data type of the hardness, elasticity, and density data types of the item, vim is the standard value of the i-th data type in the data types of the hardness, elasticity, and density of the item, Vs is the sum of the volumes of all foreign objects, ls is the average distance between foreign objects, and Rs is the road surface width.

[0061] It should be noted that the influence of the physical properties of an object on the degree of danger: Hardness: The hardness of an object directly affects the damage degree to the vehicle or road. The greater the hardness, the greater the impact force of the object on the vehicle or road, and the higher the degree of danger; Elasticity: The elasticity of an object determines its rebound ability when being impacted. The greater the elasticity, the smaller the damage; Density: The density of an object reflects its mass. The greater the density, the greater the inertia of the object, and the higher the energy during impact; Volume: The volume of an object directly affects the space it occupies on the road. The larger the volume, the greater the difficulty for the vehicle to detour or pass through; The influence of road distribution on the degree of danger: The volume and distribution density of foreign objects: The more the number of foreign objects per unit area, the higher the collision probability when the vehicle is driving, and the greater the degree of danger; The average distance between foreign objects: The smaller the distance between objects, the more limited the detour space when the vehicle is driving, and the higher the degree of danger; Road surface width: The smaller the road surface width, the more limited the space for the vehicle to avoid, and the greater the degree of danger. The formula simplifies the complex physical model through normalization and weight optimization, while retaining the key influencing factors. This design not only ensures scientificity but also facilitates calculation and optimization in practical applications;

[0062] In this embodiment, the vehicle driving danger prediction of the road based on the running condition of the vehicle and the parameter comparison between the vehicle and the foreign object includes the following specific steps:

[0063] Step 301, obtain the driving running condition of the vehicle, and conduct vehicle driving danger analysis based on the driving running condition of the vehicle. Among them, the vehicle driving danger analysis formula is: , where T is the driving time of the previous cycle, ht is the left - right swing amplitude of the vehicle during driving, and hm is the left - right swing safety amplitude of the vehicle during driving. Since most current roads are graded roads and the abnormal fluctuations of a section of road surface are roughly the same, the influence of the abnormal fluctuations of the road surface on the vehicle vibration can be ignored. Here, the influence of the abnormal fluctuations of the road surface on the swing of the vehicle is not considered. The driving danger of the vehicle after removing the influence of the road surface is its own driving danger. The swing of the vehicle easily causes a side collision with the foreign object when passing through the foreign object, thus increasing the danger of the vehicle passing through;

[0064] Step 302, obtain the driving dimension data of the vehicle and the position data of the foreign object on the road, and conduct driving abnormality analysis based on the driving dimension data of the vehicle and the position data of the foreign object on the road. Among them, the driving abnormality analysis formula is: , where Cr is the width of the vehicle, Crm is the average distance between foreign objects, Lrm is the height of the foreign object, and Lr is the chassis height of the vehicle; the vehicle width is an important parameter of the vehicle size, directly affecting the stability and passing ability of the vehicle; the distribution density of objects directly affects the safety during vehicle driving, and the greater the density, the higher the degree of danger; the height of the foreign object determines the possibility of contact between the vehicle chassis and the foreign object; the chassis height is an important parameter in vehicle design, directly affecting the safety of the vehicle when passing foreign objects. In order to normalize the parameters, the formulas and ;

[0065] Step 303: The obtained vehicle driving risk analysis result and driving anomaly analysis result are weighted and summed to obtain the vehicle driving risk prediction result;

[0066] In this embodiment, the driving anomaly prediction of the vehicle based on the obtained road anomaly risk analysis result and the vehicle driving risk prediction result of the road includes the following specific contents:

[0067] Obtain the calculated vehicle driving risk prediction result and the road anomaly risk analysis result, and multiply them to obtain the vehicle driving anomaly prediction result;

[0068] Set the driving anomaly threshold. If the driving anomaly prediction result is greater than or equal to the driving anomaly threshold, it indicates that there will be danger during vehicle driving, and the vehicle is reminded to change the route as soon as possible;

[0069] If the driving anomaly prediction result is less than the driving anomaly threshold and greater than or equal to 70% of the driving anomaly threshold, the driver is reminded to pay attention to the road conditions;

[0070] If the driving anomaly prediction result is less than 70% of the driving anomaly threshold, it indicates that the driving is safe;

[0071] In this embodiment, the reminder of danger signals based on the vehicle driving anomaly prediction result includes the following specific steps: Obtain the vehicle information traveling to this section in the next stage, and send corresponding signals of danger during driving, reminding the driver to pay attention to the road conditions or driving safety through the corresponding communication method, and at the same time, send the position and type information of the foreign objects to the road management department in real time to remind the road management department to clean up.

[0072] Here, it should be noted that the vehicle driving risk prediction result reflects the comprehensive influence of various risk factors during vehicle driving (such as vehicle speed, chassis height, foreign object width, etc.); the road abnormal risk analysis result reflects the potential risk to vehicle driving from the distribution of foreign objects in the road environment (such as foreign object height, distribution density, spacing, etc.); the physical meaning of the multiplication operation is that by multiplying the two, the coupling effect between the vehicle and the road environment can be comprehensively evaluated, so as to obtain a more accurate driving abnormality prediction result; the driving abnormality threshold: through experimental data statistics and weight optimization, a critical value for driving abnormality is set. If the prediction result reaches or exceeds this threshold, it indicates that there is a relatively high risk probability for vehicle driving; the 70% threshold: in order to distinguish different levels of danger, 70% of the driving abnormality threshold is set as a secondary warning line to remind the driver to pay attention to the road conditions; the safety threshold: if the prediction result is lower than the secondary warning line, it indicates that the vehicle driving has a relatively high safety level.

[0073] Secondly, it should be noted in this embodiment that the preferred value-taking method of the set parameters (such as thresholds, respective proportion influences, or weights) in this application is as follows: Obtain the physical characteristics and distribution of historical foreign object types and vehicle data, and at the same time obtain the judgment result of whether vehicle damage has occurred. Substitute the historical situation data into the calculation result of this embodiment and import the judgment result into the fitting software for data fitting iteration to obtain the value-taking of the set parameters (such as thresholds, respective proportion influences, or weights) that conforms to the highest judgment result accuracy rate.

[0074] Finally, here is the advantage of this embodiment. This embodiment conducts road abnormal risk analysis based on the physical characteristics and distribution of the predicted foreign object types, conducts vehicle driving risk prediction on the road based on the vehicle's operating conditions and the parameter comparison between the vehicle and the foreign object, conducts vehicle driving abnormality prediction based on the obtained road abnormal risk analysis result and the vehicle driving risk prediction result on the road, and conducts a comprehensive analysis based on the physical characteristics and distribution of the predicted foreign object types, combined with the vehicle's operating state and the parameter comparison between the vehicle and the foreign object, which can more comprehensively and accurately predict the vehicle's driving abnormality, so as to significantly improve road safety.

[0075] Secondly, as Figure 4As shown in the figure, this embodiment also provides an intelligent management system for foreign objects on highway pavements based on image analysis, which is implemented based on the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis. Specifically, it includes a foreign object type analysis module, a road anomaly and danger analysis module, a vehicle driving danger prediction module, a driving anomaly prediction module, and a signal reminder module. Among them, the foreign object type analysis module obtains the image information of foreign objects on the highway pavement and simultaneously obtains the data of passing vehicles on the road for the analysis of foreign object types. The road anomaly and danger analysis module conducts road anomaly and danger analysis based on the physical characteristics and distribution of the predicted foreign object types. The vehicle driving danger prediction module predicts the vehicle driving danger on the road based on the running conditions of the vehicle and the comparison of vehicle and foreign object parameters. The driving anomaly prediction module predicts the driving anomaly of the vehicle based on the obtained road anomaly and danger analysis results and the vehicle driving danger prediction results on the road. The signal reminder module gives a reminder of danger signals based on the vehicle driving anomaly prediction results. It should be noted that Figure 4 the arrow direction in

[0076] Then, this embodiment also provides an electronic device, including: a processor and a memory. Among them, the memory stores a computer program that can be called by the processor;

[0077] The processor executes the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis by calling the computer program stored in the memory.

[0078] This electronic device may have relatively large differences due to different configurations or performances, and can include one or more processors and one or more memories. Among them, the memory stores at least one computer program, and this computer program is loaded and executed by the processor to implement the intelligent management method for foreign objects on highway pavements based on image analysis provided by the above method embodiment. This electronic device can also include other components for realizing the functions of the device. For example, this electronic device can also have components such as wired or wireless network interfaces and input / output interfaces for data input and output. This embodiment will not be elaborated here.

[0079] Finally, this embodiment proposes a computer-readable storage medium, on which a rewritable computer program is stored;

[0080] When the computer program runs on a computer device, it enables the computer device to execute the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis.

[0081] For example, the computer-readable storage medium can be a read-only memory, a random access memory, a read-only optical disc, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0082] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0083] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0084] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0086] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0087] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0088] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0089] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0090] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. An intelligent management method for foreign objects on highway pavements based on image analysis, characterized in that, It includes the following specific steps: Step 1: Obtain the image information of foreign objects on the road surface of the highway, and at the same time obtain the vehicle data passing through the road for the analysis of the types of foreign objects. The specific content is as follows: Step 101: Obtain the image information of foreign objects on the road surface of the highway through the image acquisition terminal. At the same time, obtain the additional attachments and transported goods information reported by the vehicles traveling from the position of the road foreign objects at each moment, and at the same time obtain the passing duration and passing distance of the traveling vehicles. Step 102: Conduct an item similarity analysis based on the image information of foreign objects on the road surface of the highway, the transported goods of the vehicle, and the additional attachment information. Step 103: Obtain the transported goods of the corresponding vehicles whose cosine similarity is greater than or equal to the set similarity threshold, obtain the distances and durations from the driving positions of these vehicles to the appearance of the foreign object, and calculate the selection value based on the distances and durations from the driving positions of the vehicles to the appearance of the foreign object and the cosine similarity between the foreign object image and the transported goods of the vehicles. Among them, the calculation formula for the selection value of the i-th transported good of the d-th vehicle is: , where Lm is the set standard distance, Ldi is the minimum distance from the driving position of the vehicle to the appearance of the foreign object, tm is the set standard duration, tdi is the duration from the vehicle driving to the corresponding driving position to the appearance of the foreign object, and set the type of the transported good corresponding to the maximum selection value as the type of the foreign object, S di is the cosine similarity between the foreign object image and the i-th transported good of the d-th vehicle; Step 2: Conduct an analysis of the abnormal danger of the road based on the physical characteristics and distribution of the predicted types of foreign objects. Step 3: Predict the driving danger of vehicles on the road based on the running conditions of the vehicles and the parameter comparison between the vehicles and the foreign objects. Step 4: Predict the abnormal driving of vehicles based on the obtained analysis results of the abnormal danger of the road and the predicted results of the driving danger of vehicles on the road. Step 5: Remind of the danger signal based on the predicted results of the abnormal driving of vehicles.

2. The intelligent management method for foreign objects on highway pavement based on image analysis according to claim 1, characterized in that, The analysis of the abnormal danger of the road based on the physical characteristics and distribution of the predicted types of foreign objects includes the following specific steps: Step 201: Obtain the hardness, elasticity, and density of the items corresponding to the types of foreign objects, and at the same time obtain the volume of the foreign objects and their distribution on the road. Step 202: Analyze the road hazard situation based on the hardness, elasticity, and density of the items corresponding to the foreign object types, as well as the volume of the foreign objects and their distribution on the road. Among them, the road abnormal hazard analysis formula is: , where M is the number of data types of the hardness, elasticity, and density of the items, gi is the influence weight of the i-th data type in the data types of the hardness, elasticity, and density of the items on the impact hazard, vi is the data value of the i-th data type in the i-th data type of the hardness, elasticity, and density of the items, vim is the standard value of the i-th data type in the data types of the hardness, elasticity, and density of the items, Vs is the sum of the volumes of all foreign objects, ls is the average distance between foreign objects, and Rs is the road surface width.

3. The intelligent management method for foreign objects on highway pavement based on image analysis according to claim 2, characterized in that, The prediction of the driving danger of vehicles on the road based on the running conditions of the vehicles and the parameter comparison between the vehicles and the foreign objects includes the following specific steps: Step 301: Obtain the driving operation condition of the vehicle, and conduct vehicle driving risk analysis based on the driving operation condition of the vehicle. The vehicle driving risk analysis formula is as follows: , where T is the driving time in the previous cycle, ht is the left - right swing amplitude of the vehicle during driving, and hm is the left - right swing safety amplitude of the vehicle during driving; Step 302: Obtain the driving dimension data of the vehicle and the position data of the foreign object on the road, and perform driving anomaly analysis based on the driving dimension data of the vehicle and the position data of the foreign object on the road. The driving anomaly analysis formula is as follows: , where Cr is the width of the vehicle, Crm is the average distance between foreign objects, Lrm is the height of the foreign object, and Lr is the chassis height of the vehicle; Step 303: Obtain the obtained analysis results of the driving danger of vehicles and the analysis results of the abnormal driving, and perform weighted summation to obtain the predicted results of the driving danger of vehicles.

4. The intelligent management method for foreign objects on highway pavement based on image analysis according to claim 3, characterized in that, The prediction of the abnormal driving of vehicles based on the obtained analysis results of the abnormal danger of the road and the predicted results of the driving danger of vehicles on the road includes the following specific content: Obtain the predicted results of the driving danger of vehicles and the analysis results of the abnormal danger of the road obtained by calculation, and multiply them to obtain the predicted results of the abnormal driving of vehicles. Set the abnormal driving threshold. If the predicted result of the abnormal driving is greater than or equal to the abnormal driving threshold, it indicates that there will be danger in the vehicle driving, and remind the vehicle to change the road as soon as possible. If the predicted result of the abnormal driving is less than the abnormal driving threshold and greater than or equal to 70% of the abnormal driving threshold, remind the driver to pay attention to the road conditions. If the predicted result of the abnormal driving is less than 70% of the abnormal driving threshold, it means that the driving is safe.

5. The intelligent management method for foreign objects on highway pavements based on image analysis according to claim 4, characterized in that, The reminder of the danger signal based on the predicted results of the abnormal driving of vehicles includes the following specific steps: Obtain the vehicle information traveling to this section in the next stage, and send the corresponding signals of driving danger, reminding the driver to pay attention to the road conditions or driving safety to the vehicle driver through the corresponding communication method. At the same time, send the position and type information of the foreign objects to the road management department in real time to remind the road management department to clean them up.

6. The intelligent management method for foreign objects on highway pavements based on image analysis according to claim 1, characterized in that, The method for analyzing the similarity of the articles is as follows: obtaining the shape information of the corresponding foreign object image and the color information of each point to construct the image feature vector of the foreign object image, obtaining the shape information and color information of the transported goods and external attachments of each vehicle to construct the image feature vector of the transported goods and external attachment image, and importing the image feature vector of the foreign object image and the image feature vectors of the transported goods and external attachment images into the cosine similarity calculation formula to calculate the cosine similarity between the foreign object image and each transported object of each vehicle. Among them, the cosine similarity calculation formula between the foreign object image and the i-th transported object of the d-th vehicle is: , where is the image feature vector of the foreign object image, is the feature vector of the i-th transported object of the d-th vehicle, is the modulus of the vector.

7. An intelligent management system for foreign objects on highway pavements based on image analysis, which is implemented based on the intelligent management method for foreign objects on highway pavements based on image analysis according to any one of claims 1-6, characterized in that, Specifically, it includes a foreign object type analysis module, a road abnormal danger analysis module, a vehicle driving danger prediction module, a driving abnormality prediction module, and a signal reminder module; among them, the foreign object type analysis module obtains the image information of foreign objects on the road surface of the highway and simultaneously obtains the vehicle data passing through the road for the analysis of foreign object types; the road abnormal danger analysis module conducts road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types; the vehicle driving danger prediction module conducts vehicle driving danger prediction on the road based on the operating conditions of the vehicle and the parameter comparison between the vehicle and the foreign object; the driving abnormality prediction module conducts vehicle driving abnormality prediction based on the obtained road abnormal danger analysis result and the vehicle driving danger prediction result on the road; the signal reminder module gives a reminder of danger signals based on the vehicle driving abnormality prediction result.

8. An electronic device, comprising: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; It is characterized in that the processor executes the intelligent management method for foreign objects on the road surface of the highway based on image analysis according to any one of claims 1-6 by calling the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that, Stored with instructions, when the instructions run on a computer, the computer is caused to execute the intelligent management method for foreign objects on the road surface of the highway based on image analysis according to any one of claims 1-6.

Images