Highway pavement foreign matter intelligent management method and system based on image analysis

Abstract

The invention discloses a highway pavement foreign matter intelligent management method and a highway pavement foreign matter intelligent management system based on image analysis, and belongs to the field of image analysis. The method comprises the following steps: performing vehicle driving danger prediction on a road based on the running condition of the vehicle and the parameter comparison condition of the vehicle and the foreign matter, performing vehicle driving abnormity prediction on the basis of an obtained road abnormity danger analysis result and a road vehicle driving danger prediction result, and performing vehicle driving abnormity prediction on the basis of the predicted physical characteristics and distribution condition of foreign matter types. The running state of the vehicle and the parameter comparison of the vehicle and the foreign matter are combined for comprehensive analysis, so that the running abnormity of the vehicle can be predicted more comprehensively and accurately, and the road safety is remarkably improved.

Description

Technical Field

[0001] This application belongs to the field of image analysis, specifically an intelligent management method and system for foreign objects on highway pavements based on image analysis. Background Art

[0002] Road traffic safety is one of the key concerns in modern society, and road abnormal danger analysis and vehicle driving danger prediction are the key technologies to ensure road traffic safety. With the acceleration of logistics transportation and urbanization, the number of transport vehicles and foreign objects on the road is increasing continuously. How to accurately identify and predict road abnormal dangers has become an important issue in improving road traffic safety. In the prior art, road abnormal danger analysis and cargo identification mainly rely on the similarity analysis between transported goods and foreign objects. This method judges the type of foreign objects by comparing the appearance features of goods and foreign objects, such as color and size. However, this method has obvious limitations. For example, when the goods transported by two vehicles within a certain time period are highly similar in color and size, such as iron balls and rubber ball models, it is difficult for the prior art to accurately distinguish them, resulting in identification errors and unable to effectively predict potential dangers. In addition, single similarity analysis cannot comprehensively reflect the impact of the physical properties of foreign objects on driving safety, such as the hardness, elasticity, and density of foreign objects. These physical properties are directly related to the damage degree and rebound possibility of foreign objects when they are hit by vehicles. Therefore, the prior art has the problem of insufficient prediction accuracy in practical applications and cannot meet the requirements of intelligent driving and logistics transportation for high-precision danger prediction. In view of this, this application designs an intelligent management method and system for foreign objects on highway pavements based on image analysis. Summary of the Invention

[0003] To solve the deficiencies in the prior art mentioned in the background art, this application proposes an intelligent management method and system for foreign objects on highway pavements based on image analysis. This application conducts road abnormal danger analysis based on the physical properties and distribution of the predicted foreign object types, conducts road vehicle driving danger prediction based on the running conditions of the vehicle and the parameter comparison between the vehicle and foreign objects, conducts vehicle driving abnormality prediction based on the obtained road abnormal danger analysis results and road vehicle driving danger prediction results, and conducts comprehensive analysis based on the physical properties and distribution of the predicted foreign object types, combined with the running state of the vehicle and the parameter comparison between the vehicle and foreign objects, which can more comprehensively and accurately predict vehicle driving abnormalities and significantly improve road safety.

[0004] To achieve the above object, this application provides the following technical solutions: In the first aspect, this application provides an intelligent management method for foreign objects on highway pavements based on image analysis, which includes the following specific steps: Step 1: Obtain the image information of foreign objects on the road surface of the highway, and at the same time obtain the data of passing vehicles on the road for the analysis of the types of foreign objects; Step 2: Conduct an analysis of road abnormal danger based on the physical characteristics and distribution of the predicted types of foreign objects; Step 3: Predict the driving danger of vehicles on the road based on the running conditions of the vehicles and the comparison of the parameters between the vehicles and the foreign objects; Step 4: Predict the abnormal driving of vehicles based on the obtained results of the road abnormal danger analysis and the predicted results of the driving danger of vehicles on the road; Step 5: Give a reminder of the danger signal based on the predicted results of the abnormal driving of vehicles.

[0005] Preferably, based on the above solution, the specific content of obtaining the image information of foreign objects on the road surface of the highway and at the same time obtaining the data of passing vehicles on the road for the analysis of the types of foreign objects is as follows: Step 101: Obtain the image information of foreign objects on the road surface of the highway through an image acquisition terminal, including the shape information and color information of the corresponding image. At the same time, obtain the information of external attachments and transported goods reported by the vehicles driving from the position of the road foreign objects at each moment. Among them, the information of transported goods and external attachments includes the types of items, the shape information and color information of the items, and also includes the material characteristic data of the items such as the hardness, elasticity, and density of the items; at the same time, obtain the passing duration and passing distance of the driving vehicles; Step 102: Conduct an analysis of the similarity of items based on the image information of foreign objects on the road surface of the highway and the information of transported goods and external attachments of the vehicles. The method of analyzing the similarity of items is as follows: Obtain the shape information of the corresponding foreign object image and the color information of each point to construct the image feature vector of the foreign object image. Obtain the shape information and color information of the transported goods and external attachments of each vehicle to construct the image feature vector of the transported goods and external attachments image. Import the image feature vector of the foreign object image and the image feature vectors of the transported goods and external attachments images into the cosine similarity calculation formula to calculate the cosine similarity between the foreign object image and each transported item of each vehicle. Among them, the cosine similarity calculation formula between the foreign object image and the i-th transported item of the d-th vehicle is: , where, is the image feature vector of the foreign object image, is the feature vector of the i-th transported item of the d-th vehicle, is the modulus of the vector; Step 103: Obtain the transported goods of the corresponding vehicles with a cosine similarity greater than or equal to the set similarity threshold. Obtain the distances and durations from the driving positions of these vehicles to the appearance of the foreign object. Calculate the selection value based on the distances and durations from the driving positions of the vehicles to the appearance of the foreign object and the cosine similarity between the foreign object image and the transported goods of the vehicles. Among them, the calculation formula for the selection value of the i-th transported good of the d-th vehicle is: , where Lm is the set standard distance, Ldi is the minimum distance from the driving position of the vehicle to the appearance of the foreign object, tm is the set standard duration, tdi is the duration from the vehicle driving to the corresponding driving position to the appearance of the foreign object. Set the type of transported good corresponding to the maximum selection value as the type of foreign object; Preferably, based on the above solution, the road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types includes the following specific steps: Step 201: Obtain the hardness, elasticity, and density of the items corresponding to the foreign object types. At the same time, obtain the volume of the foreign object and its distribution on the road; Step 202: Analyze the road danger situation based on the hardness, elasticity, and density of the items corresponding to the foreign object types, the volume of the foreign object, and its distribution on the road. Among them, the road abnormal danger analysis formula is: , where M is the number of data types of the hardness, elasticity, and density of the items, gi is the influence weight of the i-th data type in the data types of the hardness, elasticity, and density of the items on the impact danger. Collect the danger degree data of different foreign objects under different conditions through experiments. According to the experimental data, adjust the weight coefficient through machine learning algorithms (such as regression analysis or neural network). vi is the data value of the i-th data type in the i-th data type of the hardness, elasticity, and density data types of the items, vim is the standard value of the i-th data type in the data types of the hardness, elasticity, and density of the items. Vs is the sum of the volumes of all foreign objects, ls is the average distance between foreign objects, and Rs is the road surface width.

[0006] Preferably, based on the above solution, the vehicle driving danger prediction on the road based on the vehicle running conditions and the parameter comparison between the vehicle and the foreign object includes the following specific steps: Step 301: Obtain the driving running conditions of the vehicle. Analyze the vehicle driving danger based on the driving running conditions of the vehicle. Among them, the vehicle driving danger analysis formula is: , where T is the driving time of the previous cycle, ht is the left and right swing amplitude of the vehicle during driving, and hm is the left and right swing safety amplitude of the vehicle during driving; Step 302: Obtain the driving dimension data of the vehicle and the position data of the foreign object on the road. Analyze the driving abnormality based on the driving dimension data of the vehicle and the position data of the foreign object on the road. Among them, the driving abnormality analysis formula is: , where Cr is the width of the vehicle, Crm is the average distance between foreign objects, Lrm is the height of the foreign object, and Lr is the chassis height of the vehicle; the vehicle width is an important parameter of the vehicle size, directly affecting the stability and passing ability of the vehicle; the distribution density of objects directly affects the safety when the vehicle is driving, and the greater the density, the higher the degree of danger; the height of the foreign object determines the possibility of contact between the vehicle chassis and the foreign object; the chassis height is an important parameter in vehicle design, directly affecting the safety of the vehicle when passing through foreign objects. In order to normalize the parameters, the formulas and ; Step 303: The vehicle driving risk prediction result is obtained by performing weighted summation on the obtained vehicle driving risk analysis result and driving anomaly analysis result.

[0007] Preferably, based on the above solution, the driving anomaly prediction of the vehicle based on the obtained road anomaly risk analysis result and the vehicle driving risk prediction result of the road includes the following specific contents: Obtain the calculated vehicle driving risk prediction result and road anomaly risk analysis result, and multiply them to obtain the vehicle driving anomaly prediction result; Set the driving anomaly threshold. If the driving anomaly prediction result is greater than or equal to the driving anomaly threshold, it indicates that the vehicle will be in danger during driving, and the vehicle is reminded to change the route as early as possible; If the driving anomaly prediction result is less than the driving anomaly threshold and greater than or equal to 70% of the driving anomaly threshold, the driver is reminded to pay attention to the road conditions; If the driving anomaly prediction result is less than 70% of the driving anomaly threshold, it indicates that the driving is safe.

[0008] Preferably, based on the above solution, the reminder of the danger signal based on the vehicle driving anomaly prediction result includes the following specific steps: Obtain the vehicle information driving to this section in the next stage, and send the corresponding signal of driving in danger, reminding the driver to pay attention to the road conditions or driving safely to the vehicle driver through the corresponding communication method, and at the same time, send the position and type information of the foreign object to the road management department in real time to remind the road management department to clean it up.

[0009] Second aspect, the present application provides an intelligent management system for foreign objects on highway pavements based on image analysis, which is implemented based on the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis. Specifically, it includes a foreign object type analysis module, a road anomaly danger analysis module, a vehicle driving danger prediction module, a driving anomaly prediction module, and a signal reminder module. Among them, the foreign object type analysis module obtains the image information of foreign objects on the highway pavement and simultaneously obtains the vehicle data passing through the road for the analysis of foreign object types; the road anomaly danger analysis module conducts road anomaly danger analysis based on the physical characteristics and distribution of the predicted foreign object types; the vehicle driving danger prediction module conducts vehicle driving danger prediction on the road based on the running conditions of the vehicle and the parameter comparison between the vehicle and the foreign object; the driving anomaly prediction module conducts vehicle driving anomaly prediction based on the obtained road anomaly danger analysis results and vehicle driving danger prediction results on the road; the signal reminder module gives a reminder of danger signals based on the vehicle driving anomaly prediction results.

[0010] Third aspect, the present application provides an electronic device, including: a processor and a memory. Among them, the memory stores a computer program that can be called by the processor; The processor executes the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis by calling the computer program stored in the memory.

[0011] Fourth aspect, the present application provides a computer-readable storage medium storing instructions, which when run on a computer, cause the computer to execute the intelligent management method for foreign objects on highway pavements based on image analysis as described above.

[0012] At the same time, compared with the prior art, the technical effects and advantages of the present application are: The first advantage of the present application is that the present application obtains the image information of foreign objects on the highway pavement and simultaneously obtains the vehicle data passing through the road for the analysis of foreign object types, greatly improving the recognition accuracy of foreign objects and laying a foundation for subsequent driving anomaly analysis.

[0013] The second advantage of the present application is that the present application conducts road anomaly danger analysis based on the physical characteristics and distribution of the predicted foreign object types, conducts vehicle driving danger prediction on the road based on the running conditions of the vehicle and the parameter comparison between the vehicle and the foreign object, conducts vehicle driving anomaly prediction based on the obtained road anomaly danger analysis results and vehicle driving danger prediction results on the road, and conducts comprehensive analysis based on the physical characteristics and distribution of the predicted foreign object types, combined with the running state of the vehicle and the parameter comparison between the vehicle and the foreign object, which can more comprehensively and accurately predict vehicle driving anomalies, so as to significantly improve road safety. Description of the Drawings

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings; Figure 1 It is a schematic diagram of the overall process of the intelligent management method for foreign objects on the highway pavement based on image analysis; Figure 2 It is a schematic diagram of the specific process of step 1 of the intelligent management method for foreign objects on the highway pavement based on image analysis; Figure 3 It is a schematic diagram of the specific process of step 3 of the intelligent management method for foreign objects on the highway pavement based on image analysis; Figure 4 It is a schematic diagram of the module composition of the intelligent management system for foreign objects on the highway pavement based on image analysis. Specific embodiments

[0015] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and in no way limits the present application and its application or use.

[0016] In addition, the drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, so repeated descriptions of them will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0017] It should be understood that although the terms "first", "second", etc. may be used here to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit can be called the second unit, and similarly, the second unit can be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed related items.

[0018] To solve the technical problems proposed in the background art, the present application provides a preferred embodiment: as Figures 1-3As shown, an intelligent management method for foreign objects on a highway pavement based on image analysis includes the following specific steps: Step 1: Obtain the image information of foreign objects on the highway pavement, and at the same time obtain the vehicle data passing through the road for the analysis of the types of foreign objects; Step 2: Conduct an analysis of road abnormal hazards based on the physical characteristics and distribution of the predicted types of foreign objects; Step 3: Predict the driving hazards of vehicles on the road based on the running conditions of the vehicles and the comparison of the parameters between the vehicles and the foreign objects; Step 4: Predict the abnormal driving of vehicles based on the obtained results of road abnormal hazard analysis and the predicted results of vehicle driving hazards on the road; Step 5: Remind of danger signals based on the predicted results of abnormal vehicle driving; In this embodiment, the specific content of obtaining the image information of foreign objects on the highway pavement and at the same time obtaining the vehicle data passing through the road for the analysis of the types of foreign objects is as follows: Step 101: Obtain the image information of foreign objects on the highway pavement through an image acquisition terminal, including the shape information and color information of the corresponding image. At the same time, obtain the information of external attachments and transported goods reported by the vehicles passing through the position of the road foreign objects at each moment. Among them, the information of transported goods and external attachments includes the type of goods, the shape information and color information of the goods, and also includes the material characteristic data of the goods such as hardness, elasticity and density of the goods; at the same time, obtain the passing time and passing distance of the driving vehicles; The foreign objects on the road are usually the transported goods and external attachments that fall from the vehicles during the road transportation process. Due to the occlusion of the vehicles or the influence of the image acquisition terminal itself, the specific process of the item falling is not captured, so the type of the item is in doubt, resulting in the inability to analyze its impact on the road. It is necessary to estimate the type of the item. Therefore, it is necessary to obtain the information of external attachments and transported goods reported by the vehicles passing through the position of the road foreign objects at each moment through the vehicle management department terminal, and use the duration and distance of the vehicle driving to the appearance of the foreign object, and the similarity between the transported goods and the foreign object to jointly analyze the probability of the type of the foreign object. The prior art usually only uses the similarity between the transported goods and the foreign object to analyze the type of the foreign object. In this way, if two vehicles transport similar transported goods within a certain period of time, such as iron balls and ball models with similar colors and sizes, it will be difficult to distinguish them during recognition. And this application comprehensively uses the duration and distance of the vehicle driving to the appearance of the foreign object, that is, the shorter the distance and duration from the vehicle driving position to the appearance of the foreign object, the closer the vehicle is to the foreign object, which on the contrary indicates a greater probability that the foreign object falls from the vehicle; Step 102: Conduct an analysis of the similarity of items based on the image information of foreign objects on the highway pavement, the transported goods and external attachments information of the vehicles. The method of item similarity analysis is as follows: Obtain the shape information of the corresponding foreign object image and the color information of each point to construct the image feature vector of the foreign object image. Obtain the shape information and item color information of the transported goods and external attachments of each vehicle to construct the image feature vector of the transported goods and external attachment images. Import the image feature vector of the foreign object image and the image feature vectors of the transported goods and external attachment images of each vehicle into the cosine similarity calculation formula to calculate the cosine similarity between the foreign object image and each transported object of each vehicle. Among them, the cosine similarity calculation formula between the foreign object image and the i-th transported object of the d-th vehicle is: , where is the image feature vector of the foreign object image, is the feature vector of the i-th transported object of the d-th vehicle, is the modulus of the vector; Step 103: Obtain the transported objects of the corresponding vehicles whose cosine similarity is greater than or equal to the set similarity threshold. Obtain the distance and duration from the driving position of these vehicles to the appearance of the foreign object. Calculate the selection value based on the distance and duration from the driving position of the vehicle to the appearance of the foreign object and the cosine similarity between the foreign object image and the transported object of the vehicle. Among them, the selection value calculation formula for the i-th transported object of the d-th vehicle is: , where Lm is the set standard distance, Ldi is the minimum distance from the driving position of the vehicle to the appearance of the foreign object, tm is the set standard duration, tdi is the duration from the vehicle driving to the corresponding driving position to the appearance of the foreign object. Set the transported object type corresponding to the maximum selection value as the foreign object type; Exemplarily, the similarity threshold here is set to 90%. The standard distance and standard duration here play a dual role in eliminating units and weights. The values of the standard distance and standard duration can be adjusted to adjust the influence weights of distance and duration, which are obtained according to specific experiments. If the cosine similarity between the foreign object image and each transported object of each vehicle is less than the similarity threshold, all are substituted into the selection value calculation formula to calculate the selection values of each transported object of each vehicle. If there are multiple foreign objects of the same type, Ldi takes the minimum distance from the driving position of the vehicle to the appearance of all foreign objects, and tdi takes the minimum duration from the vehicle driving to the corresponding driving position to the appearance of one of the foreign objects; In this embodiment, the road abnormal danger analysis based on the physical properties and distribution of the predicted foreign object types includes the following specific steps: Step 201: Obtain the hardness, elasticity and density of the item corresponding to the foreign object type, and at the same time obtain the volume of the foreign object and its distribution on the road; Step 202: Analyze the road danger situation based on the hardness, elasticity and density of the item corresponding to the foreign object type, the volume of the foreign object, and its distribution on the road. Among them, the road abnormal danger analysis formula is: , where M is the number of data types of the hardness, elasticity, and density of the object, gi is the influence weight of the i-th data type in the data types of the hardness, elasticity, and density of the object on the impact hazard. By experimentally collecting the hazard degree data of different foreign objects under different conditions, according to the experimental data, the weight coefficients are adjusted through machine learning algorithms (such as regression analysis or neural network). vi is the data value of the i-th data type in the i-th data type of the hardness, elasticity, and density of the object, vim is the standard value of the i-th data type in the data types of the hardness, elasticity, and density of the object, Vs is the sum of the volumes of all foreign objects, ls is the average distance between foreign objects, and Rs is the road width.

[0019] It should be noted that the influence of the physical properties of the object on the hazard degree: Hardness: The hardness of the object directly affects the degree of damage to the vehicle or the road. The greater the hardness, the greater the impact force of the object on the vehicle or the road, and the higher the hazard degree; Elasticity: The elasticity of the object determines its rebound ability when being impacted. The greater the elasticity, the smaller the damage; Density: The density of the object reflects its mass. The greater the density, the greater the inertia of the object and the higher the energy during impact; Volume: The volume of the object directly affects the space it occupies on the road. The larger the volume, the greater the difficulty for the vehicle to detour or pass; The influence of the road distribution situation on the hazard degree: The sum of the volumes and distribution density of foreign objects: The more the number of foreign objects per unit area, the higher the collision probability when the vehicle is driving, and the greater the hazard degree; The average distance between foreign objects: The smaller the distance between objects, the more limited the detour space when the vehicle is driving, and the higher the hazard degree; Road width: The smaller the road width, the more limited the avoidance space for the vehicle, and the greater the hazard degree. The formula simplifies the complex physical model through normalization and weight optimization, while retaining the key influencing factors. This design not only ensures scientificity but also facilitates calculation and optimization in practical applications; In this embodiment, the vehicle driving hazard prediction of the road based on the running situation of the vehicle and the parameter comparison situation between the vehicle and the foreign object includes the following specific steps: Step 301, obtain the driving running situation of the vehicle, and conduct vehicle driving hazard analysis based on the driving running situation of the vehicle. Among them, the vehicle driving hazard analysis formula is: , where T is the driving time of the previous cycle, ht is the left - right swing amplitude of the vehicle during driving, hm is the left - right swing safety amplitude of the vehicle during driving. Because most of the current roads are graded roads, and the abnormal fluctuations of a section of the road surface are roughly the same, the influence of the abnormal fluctuations of the road surface on the vehicle vibration can be ignored. Here, the influence of the abnormal fluctuations of the road surface on the swing of the vehicle is not considered. The driving hazard of the vehicle after removing the influence of the road surface is its own driving hazard. The swing of the vehicle easily causes the side of the vehicle to collide with the foreign object when passing through the foreign object, thus increasing the hazard of the vehicle passing through; Step 302: Obtain the driving dimension data of the vehicle and the position data of the foreign object on the road, and perform driving anomaly analysis based on the driving dimension data of the vehicle and the position data of the foreign object on the road. The driving anomaly analysis formula is as follows: , where Cr is the width of the vehicle, Crm is the average distance between foreign objects, Lrm is the height of the foreign object, and Lr is the chassis height of the vehicle. The vehicle width is an important parameter of the vehicle size, which directly affects the stability and passing ability of the vehicle. The distribution density of objects directly affects the safety of the vehicle during driving. The greater the density, the higher the degree of danger. The height of the foreign object determines the possibility of contact between the vehicle chassis and the foreign object. The chassis height is an important parameter in vehicle design, which directly affects the safety of the vehicle when passing foreign objects. In order to normalize the parameters, the formulas and are used; Step 303: Perform weighted summation on the obtained vehicle driving risk analysis result and driving anomaly analysis result to obtain the vehicle driving risk prediction result; In this embodiment, the driving anomaly prediction of the vehicle based on the obtained road anomaly risk analysis result and the vehicle driving risk prediction result of the road includes the following specific contents: Obtain the calculated vehicle driving risk prediction result and the road anomaly risk analysis result, and multiply them to obtain the vehicle driving anomaly prediction result; Set the driving anomaly threshold. If the driving anomaly prediction result is greater than or equal to the driving anomaly threshold, it indicates that the vehicle will be in danger during driving, and the vehicle is reminded to change the road as soon as possible; If the driving anomaly prediction result is less than the driving anomaly threshold and greater than or equal to 70% of the driving anomaly threshold, the driver is reminded to pay attention to the road conditions; If the driving anomaly prediction result is less than 70% of the driving anomaly threshold, it indicates that the driving is safe; In this embodiment, the reminder of the danger signal based on the vehicle driving anomaly prediction result includes the following specific steps: Obtain the vehicle information driving to this section in the next stage, and send the corresponding signal of driving in danger, reminding the driver to pay attention to the road conditions or driving safely to the vehicle driver through the corresponding communication method. At the same time, send the position and type information of the foreign object to the road management department in real time to remind the road management department to clean it up.

[0020] It should be noted here that the vehicle driving danger prediction result reflects the comprehensive influence of various risk factors during vehicle driving (such as vehicle speed, chassis height, foreign object width, etc.); the road abnormal danger analysis result reflects the potential danger to vehicle driving caused by the distribution of foreign objects in the road environment (such as foreign object height, distribution density, spacing, etc.); the physical meaning of the multiplication operation is that by multiplying the two, the coupling effect between the vehicle and the road environment can be comprehensively evaluated, so as to obtain a more accurate driving abnormality prediction result; the driving abnormality threshold: through experimental data statistics and weight optimization, the critical value of driving abnormality is set. If the prediction result reaches or exceeds this threshold, it indicates that there is a relatively high risk probability for vehicle driving; the 70% threshold: in order to distinguish different degrees of danger, 70% of the driving abnormality threshold is set as the secondary warning line to remind the driver to pay attention to the road conditions; the safety threshold: if the prediction result is lower than the secondary warning line, it indicates that the vehicle driving has a relatively high safety.

[0021] Secondly, it should be noted in this embodiment that the preferred value-taking method of the set parameters (such as thresholds, respective proportion influences, or weights) in this application is as follows: obtain the physical characteristics and distribution of historical foreign object types and vehicle data, and at the same time obtain the judgment result of whether vehicle damage has occurred. Substitute the historical situation data into the calculation result of this embodiment and import the judgment result into the fitting software for data fitting iteration to obtain the value-taking of the set parameters (such as thresholds, respective proportion influences, or weights) that conforms to the highest judgment result accuracy rate.

[0022] Finally, the advantages of this embodiment are described here. This embodiment conducts road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types, conducts vehicle driving danger prediction on the road based on the vehicle's operating conditions and the parameter comparison between the vehicle and the foreign object, conducts vehicle driving abnormality prediction based on the obtained road abnormal danger analysis result and the vehicle driving danger prediction result on the road, and conducts comprehensive analysis based on the physical characteristics and distribution of the predicted foreign object types, combined with the vehicle's operating state and the parameter comparison between the vehicle and the foreign object, which can more comprehensively and accurately predict the vehicle's driving abnormality, so as to significantly improve road safety.

[0023] Secondly, as Figure 4As shown in the figure, this embodiment also provides an intelligent management system for foreign objects on highway pavements based on image analysis, which is implemented based on the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis. Specifically, it includes a foreign object type analysis module, a road abnormal danger analysis module, a vehicle driving danger prediction module, a driving abnormality prediction module, and a signal reminder module. Among them, the foreign object type analysis module obtains the image information of foreign objects on the highway pavement and simultaneously obtains the vehicle data passing through the road for the analysis of foreign object types. The road abnormal danger analysis module conducts road abnormal danger analysis based on the physical characteristics and distribution of the predicted foreign object types. The vehicle driving danger prediction module predicts the vehicle driving danger on the road based on the vehicle's operating conditions and the parameter comparison between the vehicle and the foreign object. The driving abnormality prediction module predicts the vehicle's driving abnormality based on the obtained road abnormal danger analysis results and the vehicle driving danger prediction results on the road. The signal reminder module gives a reminder of the danger signal based on the vehicle driving abnormality prediction results. It should be noted that Figure 4 the arrow direction in represents the data transmission direction.

[0024] Then, this embodiment also provides an electronic device, including: a processor and a memory. Among them, the memory stores a computer program that can be called by the processor; The processor executes the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis by calling the computer program stored in the memory.

[0025] This electronic device may have relatively large differences due to different configurations or performances, and can include one or more processors and one or more memories. Among them, the memory stores at least one computer program, and this computer program is loaded and executed by the processor to implement the intelligent management method for foreign objects on highway pavements based on image analysis provided by the above method embodiment. This electronic device can also include other components for realizing the functions of the device. For example, this electronic device can also have components such as wired or wireless network interfaces and input / output interfaces for data input and output. This embodiment will not be elaborated here.

[0026] Finally, this embodiment proposes a computer-readable storage medium, on which a rewritable computer program is stored; When the computer program runs on a computer device, it causes the computer device to execute the above-mentioned intelligent management method for foreign objects on highway pavements based on image analysis.

[0027] For example, the computer-readable storage medium can be a read-only memory, a random access memory, a read-only optical disc, magnetic tape, floppy disk, and optical data storage device, etc.

[0028] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0029] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0030] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0031] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0032] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0033] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0034] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0035] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0036] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An intelligent management method for foreign matter on highway pavement based on image analysis, characterized in that: It includes the following specific steps: Step 1: Obtain image information of foreign objects on the road surface, and simultaneously obtain data of vehicles passing through the road to analyze the types of foreign objects; Step 2: Perform road abnormality hazard analysis based on the predicted physical characteristics and distribution of foreign matter types; Step 3: predicting the road vehicle driving hazard based on the vehicle's operating conditions and the comparison of parameters between the vehicle and the foreign object; Step 4: predicting vehicle driving abnormality based on the obtained road abnormality risk analysis results and the vehicle driving risk prediction results on the road; Step 5: Prompt a danger signal based on the prediction result of vehicle driving abnormality.

2. The method for intelligent management of foreign matter on highway pavement based on image analysis according to claim 1, characterized in that: The specific contents of obtaining the image information of foreign bodies on the road surface and simultaneously obtaining the data of vehicles passing through the road to analyze the types of foreign bodies are as follows: Step 101: Acquire image information of foreign objects on the road surface through an image acquisition terminal, and simultaneously acquire information on attached objects and transported goods reported by vehicles traveling from the location of the foreign objects on the road at each time, and simultaneously acquire the passing time and passing distance of the traveling vehicles; Step 102: perform object similarity analysis based on the image information of foreign objects on the road surface, the transported goods of the vehicle, and the information of external attachments; Step 103: Obtain the transported objects of the corresponding vehicles whose cosine similarity is greater than or equal to the set similarity threshold, obtain the distance and duration from the driving position of these vehicles to the appearance of the foreign objects, and calculate the selection value based on the distance and duration from the driving position of the vehicles to the appearance of the foreign objects, and the cosine similarity between the foreign object image and the transported objects of the vehicles, wherein the selection value calculation formula of the i-th transported object of the d-th vehicle is: , where Lm is the set standard distance, Ldi is the minimum distance from the vehicle's driving position to the appearance of foreign objects, tm is the set standard time, tdi is the time from the vehicle driving to the corresponding driving position to the appearance of foreign objects, the type of transported object corresponding to the largest selection value is set as the type of foreign object, and Sdi is the cosine similarity between the foreign object image and the i-th transported object of the d-th vehicle.

3. The method for intelligent management of foreign matter on highway pavement based on image analysis according to claim 2, characterized in that: The road abnormality hazard analysis based on the predicted physical characteristics and distribution of foreign matter types includes the following specific steps: Step 201: Obtain the hardness, elasticity and density of the object corresponding to the type of foreign matter, and simultaneously obtain the volume of the foreign matter and its distribution on the road; Step 202: Analyze the road danger situation based on the hardness, elasticity and density of the object corresponding to the type of foreign matter, the volume of the foreign matter, and the distribution of the foreign matter on the road. The road abnormal danger analysis formula is: , where M is the number of hardness, elasticity and density data types of the object, gi is the influence weight of the i-th data type in the hardness, elasticity and density data types of the object on the impact risk, vi is the data value of the i-th data type in the hardness, elasticity and density data types of the i-th object, vim is the standard value of the i-th data type in the hardness, elasticity and density data types of the object, Vs is the volume sum of all foreign objects, ls is the average distance between foreign objects, and Rs is the road width.

4. The method for intelligent management of foreign matter on highway pavement based on image analysis according to claim 3, characterized in that: The method of predicting the danger of a vehicle on a road based on the running condition of the vehicle and the comparison of parameters between the vehicle and the foreign object comprises the following specific steps: Step 301: Obtain the driving and operating conditions of the vehicle, and perform a driving hazard analysis based on the driving and operating conditions of the vehicle, wherein the driving hazard analysis formula is: , where T is the driving time of the previous cycle, ht is the left-right swing amplitude of the vehicle during driving, and hm is the left-right swing safety amplitude of the vehicle during driving; Step 302: Acquire the driving dimension data of the vehicle and the location data of the foreign object on the road, and perform driving abnormality analysis based on the driving dimension data of the vehicle and the location data of the foreign object on the road, wherein the driving abnormality analysis formula is: , where Cr is the width of the vehicle, Crm is the average distance between foreign objects, Lrm is the height of the foreign objects, and Lr is the chassis height of the vehicle; Step 303: Obtain the vehicle driving hazard analysis result and the driving abnormality analysis result, perform weighted summation, and obtain the vehicle driving hazard prediction result.

5. The method for intelligent management of foreign matter on highway pavement based on image analysis according to claim 4, characterized in that: The vehicle driving abnormality prediction based on the obtained road abnormality risk analysis result and the vehicle driving risk prediction result of the road includes the following specific contents: Obtaining the calculated vehicle driving danger prediction result and road abnormality danger analysis result, and multiplying them to obtain the vehicle driving abnormality prediction result; Set a driving abnormality threshold. If the driving abnormality prediction result is greater than or equal to the driving abnormality threshold, it indicates that the vehicle is in danger of driving, and remind the vehicle to change lanes as soon as possible. If the driving abnormality prediction result is less than the driving abnormality threshold and greater than or equal to 70% of the driving abnormality threshold, the driver is reminded to pay attention to the road conditions; If the driving abnormality prediction result is less than 70% of the driving abnormality threshold, it means driving is safe.

6. The method for intelligent management of foreign matter on highway pavement based on image analysis according to claim 5, characterized in that: The warning of danger signals based on the abnormal vehicle driving prediction results includes the following specific steps: obtaining vehicle information that will travel to the road section in the next stage, issuing corresponding signals to the vehicle driver through the corresponding communication method that will cause danger in driving and remind the driver to pay attention to road conditions or driving safety, and at the same time issuing the location and type information of foreign objects to the road management department in real time to remind the road management department to clean them up.

7. The method for intelligent management of foreign matter on highway pavement based on image analysis according to claim 2, characterized in that: The method for analyzing the similarity of objects is as follows: obtaining the shape information of the corresponding foreign object image and the color information of each point to construct the image feature vector of the foreign object image, obtaining the shape information of the transported goods and the additional object information of each vehicle and the image feature vector of the transported goods and the additional object image, and importing the image feature vector of the foreign object image and the image feature vector of the transported goods and the additional object image into the cosine similarity calculation formula to calculate the cosine similarity between the foreign object image and each transported object of each vehicle, wherein the cosine similarity calculation formula between the foreign object image and the i-th transported object of the d-th vehicle is: ,in, is the image feature vector of the foreign body image, is the feature vector of the ith transported object of the dth vehicle, is the magnitude of the vector.

8. An intelligent management system for foreign bodies on road surfaces based on image analysis, which is implemented based on the intelligent management method for foreign bodies on road surfaces based on image analysis as claimed in any one of claims 1 to 7, characterized in that: It specifically includes a foreign body type analysis module, a road abnormality risk analysis module, a vehicle driving risk prediction module, a driving abnormality prediction module and a signal reminder module; wherein the foreign body type analysis module obtains the image information of foreign bodies on the road surface, and simultaneously obtains the data of vehicles passing through the road to analyze the types of foreign bodies; the road abnormality risk analysis module performs road abnormality risk analysis based on the predicted physical characteristics and distribution of the foreign body types; the vehicle driving risk prediction module predicts the vehicle driving risk of the road based on the vehicle's operating conditions and the parameter comparison between the vehicle and the foreign body; the driving abnormality prediction module predicts the vehicle driving abnormality based on the obtained road abnormality risk analysis results and the vehicle driving risk prediction results; the signal reminder module gives a danger signal reminder based on the vehicle driving abnormality prediction results.

9. An electronic device, comprising: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; It is characterized in that the processor executes the method for intelligent management of foreign objects on highway pavement based on image analysis as described in any one of claims 1 to 7 by calling the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method for intelligent management of foreign objects on a highway pavement based on image analysis as described in any one of claims 1 to 7.

Images