Monitoring apparatus, monitoring system and monitoring method

Abstract

A monitoring apparatus (100) receives information on or have available an elevation model (210) of a traffic route (10) for mobile vehicles (20) and a variation of the wireless transmission to or from one or more mobile vehicles (10) that are on the move. The monitoring apparatus (100) performs comparison between a variation of height of a surface of the traffic route (10) included in the elevation model (210) and the variation of the wireless transmission with each other. The monitoring apparatus (100) performs at least one of the following: monitor condition of the traffic route (10), and / or condition of the mobile vehicles (20) based on the comparison.

Description

[0001] Monitoring apparatus, monitoring system and monitoring method

[0002] Field

[0003] The invention relates to a monitoring apparatus and a monitoring system and a monitoring method.

[0004] Background

[0005] Ensuring safe and efficient driving operations on the road is a critical challenge for mobile vehicles such as bicycles, motor bicycles, personal cars, vans, busses and trucks (lorries), for example. Unpredictable factors such as snow, ice, slipperiness, roughness, potholes, low temperatures and accidents etc. can significantly impact driving conditions on the road. The better the conditions are known, the drivers and / or vehicles can more effectively optimize their routes, prepare for potential dangers, and even prevent accidents thus saving human lives. Additionally, those having responsibility of the maintenance of the road can optimize their work efficiently. Although there is some information relating roads available, there is a need for improvement.

[0006] Brief description

[0007] The present invention seeks to provide an improvement in the monitoring of traffic routes and / or moving vehicles thereon.

[0008] The invention is defined by the independent claims. Embodiments are defined in the dependent claims.

[0009] If one or more of the embodiments is considered not to fall under the scope of the independent claims, such an embodiment is or such embodiments are still useful for understanding features of the invention.List of drawings

[0010] Example embodiments of the present invention are described below, by way of example only, with reference to the accompanying drawings, in which Figure 1 illustrates an example of a monitoring system;

[0011] Figure 2 illustrates an example of a elevation model of a traffic route; Figure 3 illustrates an example of a data processing arrangement; Figure 4 illustrates an example of a flow chart of the operational steps of the surface profiling and monitoring;

[0012] Figure 5 illustrates an example of image capturing and image processing of a traffic route;

[0013] Figure 6 illustrates an example of an interrelation between surface variation of the traffic route and wireless transmission; and

[0014] Figure 7 illustrates of an example of a flow chart of a monitoring method.

[0015] Description of embodiments

[0016] The following embodiments are only examples. Although the specification may refer to “an” embodiment in several locations, this does not necessarily mean that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment.

[0017] The articles “a” and “an” give a general sense of entities, structures, components, compositions, operations, functions, connections or the like in this document. Note also that singular terms may include pluralities.

[0018] Single features of different embodiments may also be combined to provide other embodiments. Furthermore, words "comprising" and "including" should be understood as not limiting the described embodiments to consist of only those features that have been mentioned and such embodiments may also contain features / structures that have not been specifically mentioned. All combinations of the embodiments are considered possible if their combination does not lead to structural or logical contradiction.It should be noted that while Figures illustrate various embodiments, they are simplified diagrams that only show some structures and / or functional entities. The connections shown in the Figures may refer to logical or physical connections. It is apparent to a person skilled in the art that the described apparatus may also comprise other functions and structures than those described in Figures and text. It should be appreciated that details of some functions, structures, and the signalling used for measurement and / or controlling are irrelevant to the actual invention. Therefore, they need not be discussed in more detail here.

[0019] This document proposes a novel method and apparatus for profiling and monitoring traffic routes such as roads by combining digital elevation models (DEM) from aerial vehicles, for example, and wireless measurement data from moving vehicles such as cars on the traffic routes. DEM may have some information on road conditions that may have been measured using wireless measurements or images of aerial cameras. The wireless measurements may be based on measurements of signals of a radio system, for example. Additionally or alternatively, condition of the mobile vehicles may be monitored. Traditionally, the monitoring of the road conditions can be done by additional sensors placed in the moving vehicles. However, this arrangement requires additional hardware, as well as access to the information of these sensors, time of measurements, place of measurement etc. making the overall process complex to observe the dynamic changes on the surface of the traffic route. The dynamic changes may relate to winter conditions due to snow accumulation, heavy rain or a storm, for example. The dynamic changes are irregularities on the surface of the traffic route.

[0020] It is possible to leverage the versatility of aerial vehicles such as drones and form a digital elevation model of the surfaces of the traffic route(s) continuously or periodically. The word “Digital Elevation Model” (DEM) is here defined generally as a model that contains information about the road. Typically, DEM contains information of numerical representation of the earth’s surface that contains height points representing the topography, as well as the method to calculate elevations between the height points. In many cases, the elevation modelis defined as a geo-referenced 3D representation of the traffic route and its surrounding environment, constructed through photogrammetric or structure-from-motion techniques(optional). This process involves extracting key points from overlapping aerial images, estimating camera positions, and creating a dense point cloud that is converted into a surface mesh or rasterized elevation map. The elevation model may contain detailed height information, surface contours, gradients, and representations of objects such as curbs, barriers, weather, temperature, and vegetation, along with metadata like ‘GPS coordinates and confidence scores. A plurality of moving vehicles on the traffic route can act as sensors and perform wireless system measurements which include information on the driving conditions such as snow density, black ice, aquaplaning, road roughness, etc. The moving vehicles on the traffic route and the aerial vehicle(s) may be connected to the wireless network via roadside radio units or other communications infrastructure. The features of these wireless transmission measurements and the like associate features of the elevation model to and / or from the moving vehicles on the traffic route are combined to form an adaptive and robust surface profile model of the traffic route. In that manner the traffic route can be monitored over time, which can be used to enhance safety and performance. Note that the aerial vehicles 200 do not need to fly continuously but it is enough to capture images of the traffic route 10 occasionally or periodically. Additionally or alternatively, condition of the mobile vehicle 20 can be monitored.

[0021] Fig. 1 illustrates an example of a monitoring system with a monitoring apparatus 100. In an embodiment, the monitoring apparatus 100 receives information on or for a digital elevation model 210 of a traffic route 10, the traffic route 10 being for mobile vehicles 20. The digital elevation model 210 includes high-resolution information on a surface irregularity profile of the traffic route 10. The digital elevation model 210 can be formed in the monitoring apparatus 100 or in a separate elevation model apparatus 104, which may be a separate unit in the monitoring system. The digital elevation model apparatus 104 may be a part of at least one of the mobile vehicle 20, the monitoring apparatus 100, a roadside unit 150 or one or more aerial vehicles 200.In an embodiment, the monitoring apparatus 100 has the digital elevation model 210 in the memory or it is available from an outside service. The received information may be an update to an already existing elevation model 210. Alternatively, it may be so that no elevation model 210 existed prior to the reception of the information on or for the elevation model 210. In such a case, DEM may not be necessarily initially available or may be empty, scarce, or incomplete on elevation information, especially for the parts related to dynamic changes in the road condition, including potholes, etc. DEM may be just a map or location information without actual elevation information available. It may be useful to place the wireless measurements on such a map or model and use them to monitor the road condition.

[0022] In an embodiment, the monitoring apparatus 100 has available spatial information on surface roughness of a traffic route 10.

[0023] The information on or for the traffic route 10 for the digital elevation model 210 or for the spatial information can be gathered much beforehand, maybe months earlier, or only some seconds before the utilization by the monitoring apparatus 100 or it may be available in real time.

[0024] In an embodiment, the monitoring apparatus 100 and / or the elevation model apparatus 104 may form the elevation model 210 from the images of video(s) of the traffic route 10. The traffic route 10 may refer to a road or some longitudinal, bandlike structure of a terrain on which mobile vehicles 20 regularly move. The vehicles 20 move typically in an ordered manner on the traffic route 10. The monitored traffic route 10 may mean a section of a complex of traffic routes, the section being a longitudinal section. The longitudinal section may be a hundred meters to kilometers long or longer. The length of the section has no particular limit. The width of the traffic route may be meters to tens of meters, for example. The traffic route 10 for mobile vehicles 20 can be a way between places. In an embodiment, the surface of the traffic route 10 may be of the same soil as the surface of its environment. Alternatively, the surface of the traffic route 10 may be specially prepared such that it can endure traffic for a long time. Examples of such surface materials are gravel, stones, cobblestones, concrete and asphalt. The trafficroute 10 may be a highway, driveway, expressway, village road, district road, alley, street, brick road, tunnel, bridge, for example. In addition to mobile vehicles, persons and / or animals may walk on the traffic route 10.

[0025] The mobile vehicle 20 refers to a bike, a moped, an invalid carriage, a motorcycle, a personal car, a bus, an automobile, a tractor, a van, a truck, a lorry, a trailer or the like. The traffic route 20 may have or get over time one or more dangerous or harmful areas 30. The dangerous or harmful areas 30 may have snow, ice, potholes, accidents or the like that cause danger, trouble or nuisance to those who travel on the traffic route 10. Snow, ice and potholes are irregularities of the surface of the traffic route. Rain and water, in turn, may cause irregularities to the surface of the traffic route 10 while water alone may be detected as an irregularity of the surface of the traffic route 10.

[0026] An example of the elevation model 210 of a certain cross section at a certain location of the traffic route 10 is illustrated in Fig. 2. In Fig. 2, it is the question of a highway as an example. Based on the digital elevation model (DEM) assessment, it is possible to determine the surface profile. The model of Fig. 2 is formed in the wintertime. The model in Fig. 2 shows a relatively thin layer of snow on the inside lanes, which can be referred to as overtaking lanes and are used in a lesser degree. In Fig.2, the highway is divided into two roads with a central area in between. The central area has a larger amount of snow because it has not been cleared from snow like the roads themselves. Each road may have two lanes for the traffic in either direction. There may be a significant variation in the width of the road, which had an impact on the cross-section design features. The arrow markings show the lateral edge of the driveway.

[0027] The monitoring apparatus 100 measures the wireless transmission from the mobile vehicle 20 and / or receives information on wireless transmission to the mobile vehicles 20 that move on the traffic route 10. Then, the mobile vehicle 20 transmits the information to the monitoring apparatus 100. The vehicles 20 typically travel one after the other in one direction or in opposite directions and are involved with wireless transmission. The wireless transmission may be communication between mobile phones and a radio system.In an embodiment, the monitoring apparatus 100, which may comprise the sensor apparatus 102, may receive the wireless transmission transmitted by the mobile vehicles 20. The monitoring apparatus then can detect variation of wireless transmission while the mobile vehicle 20 is travelling on the traffic route 10. The communication may include V2X (Vehicle-to-Everything) and it may mean V2V communication (Vehicle to Vehicle). This technology allows vehicles to communicate with entities in the vicinity. The entities may include other vehicles and non-moving objects such as traffic lights and road signs.

[0028] In an embodiment, the sensor apparatus 102, which may be within or integrated with a mobile vehicle 20, may receive wireless transmission of the radio system. In the case the sensor apparatus 102 is not a part of the monitoring apparatus 100, the sensor apparatus 102 transmits information on the variation of the wireless transmission to the monitoring apparatus 100. The sensor apparatus 102 that is travelling with the mobile vehicle 20 may be a mobile phone or a base station of the radio system, for example.The monitoring apparatus 100 performs comparison between variation of height of a surface irregularity profile of the traffic route 10 included in the elevation model 210 and a variation of the wireless transmission with each other. Then, the monitoring apparatus 100 performs at least one of the following: monitor condition of the traffic route 10, and / or condition of the mobile vehicles 20 based on the comparison.

[0029] All in all, the monitoring apparatus 100 receives spatial information on surface roughness of a traffic route 10 and information on a variation of a wireless transmission to or from one or more mobile vehicles 10 that are on the move on the traffic route 10. Alternatively, the monitoring apparatus 100 has available spatial information on surface roughness of a traffic route 10 and receives information on a variation of a wireless transmission to or from one or more mobile vehicles 10 that are on the move on the traffic route 10.

[0030] The monitoring apparatus 100 then performs comparison between a variation of the wireless transmission and dynamic changes included in the spatial information with each other. Alternatively, the monitoring apparatus 100 thenperforms comparison between a variation of the wireless transmission and dynamic changes included in the digital elevation model 210 with each other.

[0031] The variation of the wireless transmission is caused by micromovements of the one or more mobile vehicles 20 due to an uneven surface of the traffic route and / or mechanical parts of the one or more mobile vehicles 20.

[0032] The monitoring apparatus 100 performs at least one of the following: monitor condition of the traffic route 10, and / or condition of the mobile vehicles 20 based on the comparison.

[0033] Both the spatial information and the high-resolution digital elevation model 210 have georeferenced dataset that describes the small-scale variations of a surface across traffic route 10. Both of them may measure micro-topography of the traffic route 10. In an embodiment, the monitoring apparatus 100 may perform the comparison based on an estimated effect of the variation of height of the surface irregularity profile of the traffic route 10 on the wireless transmission. A phase shift or its effect to the wireless transmission can be estimated based on a change in the propagation length of the wireless transmission caused by the variation of the height of the surface irregularity profile of the traffic route 10.

[0034] In an embodiment, the monitoring apparatus 100 may compare the variation of height of the surface of the irregularity profile of the traffic route 10 included in the elevation model 210 and the variation of the wireless transmission with each other based on similarity. Note that the variation of the height may not be explicit, and updated to the previous DEM based on the variation of the wireless transmission. This can be an example of the spatial information which is not exactly what the high-resolution DEM may be understood to be. Still, both the spatial information and the high-resolution DEM can be considered similar and in some cases equivalent. These two have little difference in principal. However, the high-resolution DEM is usually measured using LIDAR and the measurement based on radio transmissions is technically different although it gives corresponding information on road and / or vehicle condition. In this way, it is possible that the comparison is being made between the two variations of the wireless transmissions observed at different times, making monitoring possible. In somecases, this kind of comparison can be considered self-comparison. This means that the variation of the height in the digital elevation model may not have an explicit distance unit value (e.g. meters), but it may be expressed in terms of variation of the measured wireless transmission, which describes the road condition indirectly. This may also be referred to as self-comparison. By comparing two variations measured at different time instants together, it is possible to determine whether the variation has changed over time and thus monitor the road. Similarity may be measured based on a similarity metric. A person skilled in the art is familiar with the measurement of similarity, perse. Additionally or alternatively, the monitoring apparatus 100 may correlate the variation of height of the surface of the irregularity profile of the traffic route 10 included in the digital elevation model 210 and the variation of the wireless transmission with each for performing the comparison.

[0035] In an embodiment, the monitoring apparatus 100 may share information on the condition of the traffic route 10 or the condition of the mobile vehicles 20 for at least one of the following: traffic management, maintenance of the traffic route 10, vehicles 20 on the traffic route 10 and / or coming to the traffic route 10, one or more owners / drivers / passengers of the vehicles 20 on the traffic route 10, communication devices on the vehicles 20, one or more users of the traffic route for the vehicles 20 and public authorities. It is possible to share the information on the condition of the traffic route 10 and / or the vehicles 20 by sending the information in a wired or wireless manner. Alternatively or additionally, it is possible to share the information on the condition by storing it in a server memory 402 where it can be accessed by any suitable authority, organization, institution, group of humans or vehicles 20. In an embodiment, the information may be shared in multiple different ways including servers, network, wireless communications.

[0036] In an embodiment, the monitoring apparatus 100 may locate the vehicles 20 at a location within the digital elevation model 210 as function of time based on the comparison between the variation of height of the surface of the irregularity profile of the traffic route 10 included in the digital elevation model210 and the variation of the wireless transmission. Additionally or alternatively, an additional positioning system information on the locations of the mobile vehicles 20 may be utilized. The additional positioning system may be a positioning system of a radio system and / or satellite positioning system, for example. A person skilled in the art is familiar with various positioning systems.

[0037] In an embodiment, the monitoring apparatus 100 may estimate development of the condition of the traffic route 10 over time based on differences of comparisons of the variation of height of the surface of the irregularity profile of the traffic route 10 included in the digital elevation model 210 and the variation of the wireless transmission with each other as a function of time. For example, potholes may appear on the road over time and DEM created during time when the pothole did not exist or was different (for example, smaller or larger) does not thus include the current condition of the road. However, such a drastic change in road condition such as a pothole change may be seen very well in the variation of the wireless transmission. For example, potholes may cause significant variations in the phase information of the wireless transmission that can be identified based on the variation of the phase. Such potholes may be also fixed by maintenance, when the impact of pothole may disappear from the measured wireless transmission. In this way, it is clear that the system can provide up to date information of the road condition and update the DEM. Also weather can have an impact on the road conditions. For example, it may snow and the thickness of snow increases. Then the monitoring system may inform the snow cleaners to start their work. Alternatively, the snow may melt and the layer of the snow on the traffic route 10 becomes then thinner and potentially more slippery, for example. Then, the monitoring system may inform the maintenance workers that sand and / or salt should be spread on the ice or snow on the traffic route 10. Similarly, the maintenance workers may clean the snow from different parts of the road during different times. In this way, it is clear that the condition of the road may significantly vary over time, which can be monitored by the system, thus monitoring also the progress of the road maintenance.In an embodiment an example of which is illustrated in Fig. 6, the monitoring apparatus 100 may estimate speed of one or more vehicles 20 on the traffic route 10 based on the correlation between the variation of height of the surface of the irregularity profile of the traffic route 10 included in the digital elevation model 210 and the variation of the wireless communication. The speed may be estimated based on the time difference of the two or more known positions in the DEM where the variation of the height of the surface the irregularity profile on the traffic route 10 has been identified from the variation of the wireless communication.

[0038] In an embodiment, the monitoring apparatus 100 may estimate the number of the vehicles 20 passing a point of the traffic route 10 per time unit based on the comparison.

[0039] In an embodiment an example of which is illustrated in Fig. 3, the monitoring system comprises a data processing arrangement 50. The data processing arrangement 50, which may be a part of the monitoring apparatus 100, may receive information on wireless communication from mobile vehicles 20 that move on traffic route 10. In that case, the sensor arrangement 102 for detection of the variation of the wireless transmission is travelling with the mobile vehicles 20 and detecting certain features of the wireless transmission at the reception. The sensor arrangement 102 may then be a mobile phone, for example.

[0040] The data processing arrangement 50 may have the digital elevation model 210 of the traffic route 10 available and / or the data processing arrangement 50 may receive information on or for the digital elevation model 210.

[0041] The data processing arrangement 50 may perform comparison between a variation of height of a surface of the irregularity profile of the traffic route 10 included in the digital elevation model 210 and a variation of the wireless transmission with each other.

[0042] The data processing arrangement 50 may perform at least one of the following: monitor condition of the traffic route 10 where the vehicles 20 travel, and condition of the vehicles 20 based on the comparison.In an embodiment an example of which is illustrated in Fig. 1, one or more aerial vehicles 200 measure the traffic route 10, and transmit wirelessly the information on or for the digital elevation model 210 of the traffic route 10.

[0043] In an embodiment, at least one of the one or more aerial vehicles 200 comprises at least one of the following: a crewed aerial vehicle, an uncrewed aerial vehicle, high-altitude platform station, high-altitude long endurance drone and balloon. The one or more aerial vehicles 200 are equipped with at least one camera 202 configured to capture pictures and / or video of the traffic route 10. The pictures and / or videos are used for forming the digital elevation model 210 somewhere in the monitoring system.

[0044] In an embodiment, the one or more aerial vehicles 200 may comprise a high-altitude platform station (HAPS). The HAPS flying high may mean a high-altitude pseudo-satellite or high-altitude platform systems. The HAPS may include an atmospheric satellite. The HAPS may be unmanned aerial vehicles (UAVs) or it may include at least one person such as a pilot and / or other personnel. The HAPS may also mean an airplane, an aerostatic airship or a balloon.

[0045] As shown in Fig. 1, the uncrewed aerial vehicle 200 equipped with high-resolution cameras 202 may be deployed to capture images of traffic route 10 surfaces. The process for implementing an uncrewed aerial vehicle like drone to obtain the digital elevation model is shown in Fig. 1. The drone of the example may operate autonomously or under remote control, allowing it to cover extensive areas efficiently. Images can be captured from various angles and heights to provide comprehensive coverage and detailed information on conditions of the road 10. Further, these images gathered from the extensive coverage area may be transformed into 3D-constructed vision. The three-dimensional digital elevation model 210 may be formed in the aerial vehicle 200, in the data processing apparatus in the mobile vehicle 20, in the separate unit of the elevation model apparatus 104, in the monitoring apparatus 100 and / or in one or more apparatuses of the monitoring system. A person skilled in the art is familiar with this kind of image processing. An exact location of the generated 3D model can be determined with georeferencing techniques, which may gather data from varioussources, such as satellite sensors, aerial cameras, and scanned maps. Several techniques for capturing images may be used to extract georeferencing data, such as coordinates, and then integrate it or generate imagery for it. To utilize the data in a geomatic application, georeferencing is required for said monitoring condition of the traffic route 10, or the condition of the vehicles 20. The geomatic application means the collection, distribution, storing, analysing, processing, and presenting geographic data.

[0046] Inertial time of the camera 202 and the satellite positioning system time, such as the global positioning system (GPS) time, should coincide or have a deterministic relation to each other for direct georeferencing to work. After making these alterations to the data, the 3D reference point cloud and the digital elevation model 210 can be generated directly. However, it may be impractical to frequently capture images of the surface of the traffic route 10. Therefore, the pictures may be captured every time observable changes are noticed in the comparison between the variation of height of a surface of the irregularity profile of the traffic route 10 of the digital elevation model 210 and a variation of the wireless transmission with each other. The observable changes refer to difference that is greater than a threshold or resolution of in height or position. For example, if a diameter of a pothole increases about 10% larger over time based on the variation of the wireless transmission, a new image of the area of the pothole can be captured. A warning for the travellers on the traffic route 10 may also be issued.

[0047] In an embodiment, at least one of the one or more aerial vehicles 200 may capture the images and / or video in various angles with respect to the traffic route 10. The images captured in various angles may enable a three-dimensional model of the elevation of the traffic route 10.

[0048] In an embodiment an example of which is shown in Fig. 3, the data processing arrangement 50, which may correspond fully or partly to the monitoring apparatus 100, comprises one or more processors 300, and one or more memories 302 including computer program code. The one or more memories 302 and the computer program code are configured to, with the one or more processors 300, cause data processing arrangement 50 at least to perform thecomparison and the at least one of the following: monitor the condition of the traffic route 10, and condition of the vehicles 20 based on the comparison. The data processing apparatus 50 also forms the digital elevation model 210 from pictures or video that it receives.

[0049] The term “computer” includes a computational device that performs logical and arithmetic operations. For example, a “computer” may comprise an electronic computational device, such as an integrated circuit, a microprocessor, a mobile computing device, a laptop computer, a tablet computer, a personal computer, or a mainframe computer. A “computer” may comprise a central processing unit, an ALU (arithmetic logic unit), a memory unit, and a control unit that controls actions of other components of the computer so that steps of a computer program are executed in a desired sequence. A “computer” may also include at least one peripheral unit that may include an auxiliary memory (such as a disk drive or flash memory), and / or may include data processing circuitry.

[0050] A user interface 304 means an input / output device and / or unit. Nonlimiting examples of a user interface include a touch screen, other electronic display screen, keyboard, mouse, microphone, handheld electronic controller, digital stylus, display screen, speaker, and / or projector for projecting a visual display.

[0051] In an embodiment, the data processing arrangement 50 may estimate the variation of the wireless communication based on a Doppler effect caused by the variation of the height of the surface of the irregularity profile of the traffic route 10 or information on the Doppler effect. While the speed of the car itself causes Doppler effect, the variation of the transmission distance cause also variations in the phase and / or amplitude of the wireless transmission. These variations may refer to so-called micro-Doppler that can be identified. The microDoppler effect refers to minor frequency or phase modulations in the radio frequence signal caused by the movements or micro-motions such as vibrations of the vehicles 20, the micro-movements being caused by the uneven surface of the traffic route and / or mechanical parts of the vehicle 20. These micro-motions can be detected as sidebands on either or both sides of the main Doppler frequency. Intime domain, these effects are similarly seen in the phase of the signal as different speeds of phase variation over different time instants. These micro-Doppler shifts, which may refer to frequency and / or phase, and their variation over time characterize the surface of the traffic route and / or the condition of the vehicle 20. That is, technical features of the traffic route and / or the vehicle 20 can be identified and / or classified. A person skilled in the art is familiar with detection of the microDoppler effect, perse, and its mechanism of origin, perse.

[0052] In more detail, the data processing arrangement 50 may estimate the variation of the wireless transmission caused by the variation of height of a surface of the irregularity profile of the traffic route 10 based on at least one of the following: a strength of a signal of the wireless transmission at reception, a power of the signal of the wireless transmission at reception, a strength indicator of the signal of the wireless transmission, an amplitude of the signal of the wireless transmission at reception, a phase of the signal of the wireless transmission at reception, a delay of the signal of the wireless transmission at reception, a quality of the signal of the wireless transmission at reception, an error of the signal of the wireless transmission at reception, a magnitude of an error vector of the signal of the wireless transmission at reception, a mean square error of a signal of the wireless transmission at reception, a bit error rate of the signal of the wireless transmission at reception, a symbol error rate of the signal of the wireless transmission at reception, a frame error rate of the signal of the wireless transmission at reception, a packet error rate of the signal of the wireless transmission at reception, a data rate of the signal of the wireless transmission at reception, a throughput of the signal of the wireless transmission, a channel state information of the signal of the wireless transmission, a Doppler shift of the wireless transmission, a Micro Doppler shift of the wireless transmission, a variation of Doppler and / or micro-doppler shift over time, a Doppler spread of the wireless transmission, a channel frequency response of the signal of the wireless transmission, a channel impulse response of the wireless transmission, a time domain channel information of the wireless transmission, at least a part of received data of the wireless transmission, a quality of at least a part of the received dataquality, and / or a variation of one or more of these alone or combined momentarily, statistically and / or as a function of time.

[0053] Any of the said features alone or in some combination and a height variation of the traffic route 10 have a deterministic correlation between each other. Because the height variation varies the distance the electromagnetic radiation travels between the transmitter and the receiver. The distance variation then varies any of said features. The variation of said features can in general be called a variation of the wireless transmission. Alternatively, the condition of the mobile vehicle(s) may cause variation of the distance the electromagnetic radiation travels between the transmitter and the receiver.

[0054] In an embodiment, the data processing arrangement 50 may estimate the variation of the wireless transmission and thus also the variation of the surface of the traffic route 10 based on a phase fluctuation of the Doppler effect in the wireless communication of an analog or digital form.

[0055] Fig. 4 illustrates an example of a flow chart of the operational steps of the surface profiling and monitoring. In step 400, images and position information is received from the one or more aerial vehicles 200 for data processing. In step 402, the digital elevation model 210 of the traffic route 10 is formed in the data processing apparatus 50 which is in some part or parts of the monitoring system. In step 404, mobile vehicles 20 are moving on the traffic route 10. In step 406, the mobile vehicles 20 receive or output wireless transmissions and the data processing apparatus 50 performs, based on the wireless transmissions, measurements of variation in the transmissions, the variation being caused by the variation of the surface of the traffic route 10 and / or the mobile vehicle 20. The variation of the wireless transmissions is synchronized with the digital elevation model 210 of the traffic route 10 in data mapping procedure 408 where the variation information of the wireless transmissions and the digital elevation model 210 is combined. This can be based on satellite positioning and / or correlation between the variation of the surface height variation of the traffic route 10 and the variation of the wireless transmission.The result is an improved digital elevation model 210 with information provided by the variation of the wireless transmissions. The digital elevation model 210 may be presented as a three-dimensional map of the traffic route 10 with potentially one or more areas 30 of dangerous or harmful surface conditions at locations shown in the three-dimensional map. Alternatively, the one or more dangerous or harmful areas 30 may be defined by alphanumerical symbols which tell people where they are. In step 310, a surface profile of the traffic route 10 may be formed. The surface profile is one form of the digital elevation model 210 and an example of such a profile is illustrated in Fig. 2. The surface profile of the traffic route 10 can be formed in an adaptive manner in step 410, because the data from the moving vehicles 20 may be continuous and also the aerial vehicles 200 may capture images of the traffic route 10 repeatedly. In step 412, it can be defined whether an adversity of the traffic route 10 has been exceeded. The adversity here refers to any kind of difficulty to travel on the traffic route 10. If it has been exceeded, a new road surface profile may be formed based on new information from the mobile vehicles 20 and / or the aerial vehicles 200. The step 410 may also output a driving hazard identification 414 if the condition of the traffic route 20 is worse than a threshold condition. The term worse means here that a numerical value of the condition is lower than that of a threshold where a good condition has larger value than a poor condition. A person skilled in the art understands what a good and poor conditions of a traffic route 10, perse, means. Any data for the data processing may be available in one or more memory 302.

[0056] All these operations may be performed in the data processing apparatus 50 in one location or distributed in a plurality of locations. In an embodiment, the operations may be performed in the roadside unit 150 or which receives the images or video(s) or the information on the digital elevation model 210 from the aerial vehicles 200 and the moving vehicles 20 on the traffic route 10. That is, the digital elevation model 210 may be available to the roadside unit 150 or the roadside unit 150 may form the digital elevation model 210. Additionally or alternatively, these operations may be performed in one or more mobile vehicles 20 on the traffic route 10. Still additionally or alternatively, the operations may beperformed in the aerial vehicles 200. It is still an alternative or an additional possibility that the operations are performed by an apparatus of the system, or the operations are distributed over a variety of apparatuses within the system which may include cloud processing.

[0057] Fig. 5 illustrates an example of steps forming information based on images from the aerial vehicles 200. In step 500, the aerial vehicles 200 may fly in the vicinity of and / or over the traffic route 10 and the aerial vehicles 200 may comprise autonomous flying devices such as drones. In step 502, the aerial vehicles 200 capture images and / or video frames and may associate positions of themselves or the target area of the traffic route 10 in the images and / or videos. In step 504, a digital elevation model 210 is formed by the data processing apparatus 50 of the monitoring system in one or more locations. The digital elevation model 210 may be a three-dimensional map of the traffic route 10. The digital elevation model 210 may be further processed into the form of a profile graph of the surface of the traffic route 10 in step 406. In step 508, one or more hazard zones 30 may be identified in the profile graph of the surface of the traffic route 10.

[0058] Fig. 6 illustrates an example of the interdependence of the variation of the height of the irregularity profile of the traffic route 10 and the variation of the wireless transmission. When two surface variations 60 such as potholes (i.e. danger or harmful area 30) of the traffic route 10 are larger than the average variation i.e. larger than a noise level variation they cause a mobile vehicle 20 to make sudden unipolar movement in a vertical direction. That movement causes a distance variation between a transceiver moving with the mobile vehicle 20 and a transceiver outside the mobile vehicle. Assuming the two transceivers communicate wirelessly with each other, the wireless transmission experiences two variation peaks 62 corresponding to the two surface variations 60. If the variations 62 of the wireless transmission become larger and larger over time, it can be interpreted that the surface variations 60 that are larger than an average also become larger and larger. Naturally, if the variations 62 of the wireless transmission become smaller, it can be interpreted that the surface variations 30 that are larger than the average become smaller, too.Because the surface variations 60 larger than an average or equal to the average have known locations and hence their distance is known or computable, it is possible to determine a speed of each mobile vehicle 20 encountering such surface variations 60 based on the time difference between the peaks 62 in the wireless transmission.

[0059] The technical solution can further be explained using the following concrete example. Wireless infrastructure such as 4G, 5G, 6G radio systems that can also be called cellular communication systems or any one or more roadside radio units 150 with radio transceivers can be mounted on traffic poles or buildings around the road infrastructures. Moreover, modern mobile vehicles such as cars are typically equipped with modems capable of communicating with wireless infrastructure such as cellular networks. The cellular network access points / base stations frequently interact with the moving vehicles to support communication to the vehicle wireless modem and mobile telecommunication devices inside the mobile vehicles 20. Such communication is impacted by vehicle moving patterns, moving traffic, road surface conditions, etc. For example, the mobile vehicle 20 moving on the traffic route 10 generates Doppler frequency shift effects during such communication which is caused by speed, acceleration and / or vibrations of the moving vehicles 20 together with the road surface conditions, etc. By extracting the Doppler effects corresponding to road surface conditions from each moving vehicle, a micro-Doppler signature corresponding to each discretized location on the surface of the traffic route 10 can be formed. Thus, each mobile vehicle 20 travelling on the traffic route 10 can act as a sensor and perform relevant wireless system measurements corresponding to the driving conditions on the traffic route 10 during the vehicle communication with a roadside radio infrastructure or the like. Alternatively, information on the measurements or variations of the wireless transmission detected at the reception can be transmitted to a separate data processing.

[0060] The wireless system measurements to infer road surface conditions correspond to information obtained from parameters such as Doppler frequency shifts, wireless signal strength indicators, phase variations of the wireless signals,channel state information, etc. These system measurements can also correspond to any relevant sensory information that assesses the road driving conditions of the moving vehicles. For example, the surface information on the traffic route 10 obtained from an optical or acceleration sensor installed in a mobile vehicle in 20 can be communicated the data processing apparatus 50, which may be in roadside unit 150 or in one or more locations of the monitoring system, for example, and mapped with the DEMs potentially in an adaptive manner. The information on the surface conditions of the traffic route 10 can be utilized for driving on the traffic route 10. Such wireless system measurements obtained at a time instant in a location, corresponds to the real-time surface condition of the traffic route 10 where the mobile vehicle 20 is located. All such measurements, corresponding to multiple locations on the road maybe integrated at the roadside unit(s) or the like. They enable surface monitoring of the traffic route 10 based on utilization of the mobile vehicles 20. The position of the mobile vehicles 20 can be estimated when mapped against the DEMs that are formed based on the information from the aerial vehicles 200. Note that communication between the mobile vehicle 20 and the wireless infrastructure may happen using any existing wireless communication infrastructure. Moreover, one should note that communication with the mobile vehicle 20 may also mean that the system communicates with a device, such as a cell phone, a smart phone or the like that travels with the mobile vehicle 20. It should be noted that from a wireless signal measurement point of view, using higher frequencies such as centimeter-wave and millimeter-wave frequencies may give more accurate information for measuring small vibrations of the mobile vehicles 20 on the traffic route 10, caused by the variations in the surface of the traffic route 10. For example, Doppler effect, and especially so-called microDoppler effect, caused by phase fluctuation of the radio channel due to the small variations in the link distances, is stronger in higher frequencies due to the smaller wavelength. One should note that centimeter-wave and millimeter-wave communications are one of the main communication frequency regions for 6G systems.Figure 7 is a flow chart of the monitoring method. In step 700, receiving or having spatial information on surface roughness of a traffic route (10) or a digital elevation model 210 of a traffic route 10 available for a data processing arrangement 50. In step 702, which is optional, information of wireless transmission to or from mobile vehicles 20 moving on the traffic route 10 may be measured by the data processing arrangement 50. In step 704, comparison between a variation of height of a surface of the irregularity profile of the traffic route 10 included in the digital elevation model 210 or the spatial information and a variation of the wireless communication to or from mobile vehicles 20 moving on a traffic route 10 is compared with each other by the data processing arrangement 50.

[0061] In step 706, at least one of the following is performed by the data processing arrangement 50 based on the comparison: monitor condition of the traffic route 10, and condition of the vehicles 20.

[0062] In step 708 the traffic route 10 is measured by one or more aerial vehicles 200. Instead of the one or more aerial vehicles 200, the traffic route 10 may be measured alternatively. That is, the measurement by the aerial vehicles 200 can be considered an option.

[0063] In step 710, the one or more aerial vehicles 200 transmit wirelessly information on or for the digital elevation model 210 of the road 10 for making it available for the data processing arrangement 50. Then the data processing arrangement 50 receives the information on or for the digital elevation model 210 of the road 10.

[0064] The method shown in Figure 7 may be implemented as a logic circuit solution or computer program. The computer program may be placed on a computer program distribution means for the distribution thereof. The computer program distribution means is readable by a data processing device, and it encodes the computer program commands, carries out the measurements, captures of images and optionally controls the processes on the basis of the measurements and images.The computer program may be distributed using a distribution medium which may be any medium readable by the controller. The medium may be a program storage medium, a memory, a software distribution package, or a compressed software package. In some cases, the distribution may be performed using at least one of the following: a near field communication signal, a short distance signal, and a telecommunications signal.

[0065] It will be obvious to a person skilled in the art that, as technology advances, the inventive concept can be implemented in various ways. The invention and its embodiments are not limited to the example embodiments described above but may vary within the scope of the claims.

Claims

What is claimed is:

1. A monitoring apparatus, c h a r a c te r i z e d in that the monitoring apparatus (100) is configured toreceive or have available spatial information on surface roughness of a traffic route (10) or a digital elevation model (210) of the traffic route (10) and receive information on a variation of a wireless transmission to or from one or more mobile vehicles (10) that are on the move on the traffic route (10), the digital elevation model (210) including high-resolution information on a surface irregularity profile of the traffic route (10);perform comparison between dynamic changes included in the spatial information or the digital elevation model (210) and a variation of the wireless transmission with each other, where the variation of the wireless transmission is caused by micro-movements of the one or more mobile vehicles (20) due to an uneven surface of the traffic route and / or mechanical parts of the one or more mobile vehicles (20); andperform at least one of the following: monitor condition of the traffic route (10), and / or condition of the mobile vehicles (20) based on the comparison.

2. The apparatus of claim 1, c h a r a ct e r i z e d in that the monitoring apparatus (100) is configured to perform the comparison based on an estimated effect of the variation of height in the surface irregularity profile of the traffic route (10) on the wireless transmission.

3. The apparatus of claim 1, c h a r a ct e r i z e d in thatthe monitoring apparatus (100) is configured to compare the variation of height of the surface irregularity profile of the traffic route (10) included in the digital elevation model (210) and the variation of the wireless communication with each other based on similarity, and / orthe monitoring apparatus (100) is configured to correlate the variation of height of the surface irregularity profile of the traffic route (10) included in thedigital elevation model (210) and the variation of the wireless communication with each other for performing the comparison.

4. The apparatus of claim 1, c h a r a c t e r i z e d in that the monitoring apparatus (100) is configured to share information on the condition of the traffic route (10) and / or the condition of the vehicles (20) for at least one of the following: traffic management, maintenance of the traffic route (10), mobile vehicles (20) on the traffic route (10) and / or coming to the traffic route (10), one or more owners / drivers / passengers of the vehicles on the traffic route (10), communication devices on the mobile vehicles (20), one or more users of the traffic route (10) and public authorities.

5. The apparatus of claim 1, c h a r a c t e r i z e d in that the monitoring apparatus (100) is configured to locate the mobile vehicles (20) at a location within the digital elevation model (210) as function of time based on the comparison between the variation of height of the surface profile of the traffic route (10) included in the digital elevation model (210) and the variation of the wireless communication.

6. The apparatus of claim 1, c h a r a c t e r i z e d in that the monitoring apparatus (100) is configured to estimate development of the condition of the traffic route (10) over time based on differences of comparisons of the variation of height of the surface of the traffic route (10) included in the digital elevation model (210) and the variation of the wireless communication with each other as a function of time.

7. The apparatus of claim 1 or 6, c h a r a c t e r i z e d in that the monitoring apparatus (100) is configured to measure the wireless transmission to and / or from the mobile vehicles (20).

8. A monitoring system, c h a r a c t e r i z e d in that the monitoring system comprises a data processing arrangement (50) configured to receiveinformation on a variation of wireless transmission to or from mobile vehicles (20) configured to travel on a traffic route (10);the data processing arrangement (50) is configured to receive or have available spatial information on surface roughness of the traffic route (10) and a digital elevation model (210) of the traffic route (10), the digital elevation model (210) including high-resolution information on a surface irregularity profile of the traffic route (10),perform comparison between dynamic changes included in the spatial information or the elevation model (210) and a variation of the wireless communication with each other, where the variation of the wireless transmission is caused by micro-movements of the one or more mobile vehicles (20) due to an uneven surface of the traffic route and / or mechanical parts of the one or more mobile vehicles (20), andperform at least one of the following: monitor condition of the traffic route (10), and / or condition of the mobile vehicles (20) based on the comparison.

9. The system of claim 8, c h a r a ct e r i z e d in that one or more aerial vehicles (200) are configured to measure the traffic route (10) and transmit wirelessly the information on or for the digital elevation model (210) of the traffic route (10).

10. The system of claim 8, c h a r a ct e r i z e d in that at least one of the one or more aerial vehicles (200) comprise at least one of the following: a crewed aerial vehicle, an uncrewed aerial vehicle, high-altitude platform station, high-altitude long endurance drone and balloon each equipped with at least one camera (202) configured to capture pictures and / or video of the traffic route (10).

11. The monitoring system of claim 8, c h a r a ct e r i z e d in that the data processing arrangement (50) comprises one or more processors (300), and one or more memories (302) including computer program code; and the one or more memories (302) and the computer program code are configured to, with the one or more processors (300), cause data processing arrangement (50) at least toperform the comparison and the at least one of the following: monitor the condition of the traffic route (10), and / or condition of the mobile vehicles (20) based on the comparison.

12. The monitoring system of claim 8, c h a r a ct e r i z e d in that the data processing arrangement (50) is configured to estimate the variation of the wireless transmission caused by the variation of height of a surface of the traffic route (10) based on a strength of a signal of the wireless transmission at reception, a power of the signal of the wireless transmission at reception, a strength indicator of the signal of the wireless transmission, an amplitude of the signal of the wireless transmission at reception, a phase of the signal of the wireless transmission at reception, a delay of the signal of the wireless transmission at reception, a quality of the signal of the wireless transmission at reception, an error of the signal of the wireless transmission at reception, a magnitude of an error vector of the signal of the wireless transmission at reception, a mean square error of a signal of the wireless transmission at reception, a bit error rate of the signal of the wireless transmission at reception, a symbol error rate of the signal of the wireless transmission at reception, a frame error rate of the signal of the wireless transmission at reception, a packet error rate of the signal of the wireless transmission at reception, a data rate of the signal of the wireless transmission at reception, a throughput of the signal of the wireless transmission, a channel state information of the signal of the wireless transmission, a Doppler shift of the wireless transmission, a Micro Doppler shift of the wireless transmission, a Doppler spread of the wireless transmission, a channel frequency response of the signal of the wireless transmission, a channel impulse response of the wireless transmission, a time domain channel information of the wireless transmission, at least a part of received data of the wireless transmission, a quality of at least a part of the received data quality, and / or a variation of one or more of these alone or combined momentarily, statistically and / or as a function of time.

13. The monitoring system of claim 8, c h a r a ct e r i z e d in that the data processing arrangement (50) is configured to estimate the variation of thewireless communication based on a phase fluctuation of the Doppler effect in the wireless communication of an analog or digital form.

14. A monitoring method, c h a r a ct e r i z e d byreceiving or having available (700), by a data processing arrangement (50), spatial information on surface roughness of a traffic route (10) or a digital elevation model (210) of the traffic route (10), the digital elevation model (210) including high-resolution information on a surface irregularity profile of the traffic route (10),;measuring (702), by the data processing arrangement (50), wireless transmission to or from mobile vehicles (20) moving on the traffic route (10), performing (704), by the data processing arrangement (50), comparison, between dynamic changes included in the spatial information or the digital elevation model (210) and a variation of the wireless communication with each other, where the variation of the wireless transmission is caused by micromovements of the one or more mobile vehicles (20) due to an uneven surface of the traffic route and / or mechanical parts of the one or more mobile vehicles (20), andperforming (706), by the data processing arrangement (50), at least one of the following monitoring based on the comparison: condition of the traffic route (10), and / or condition of the mobile vehicles (20).

15. The monitoring method of claim 14, c h a r a ct e r i z e d by measuring (708), by one or more aerial vehicles (200), the traffic route (10),transmitting (710), by the one or more aerial vehicles (200), wirelessly information on or for the digital elevation model (210) of the traffic route (10); and receiving (700) the information on or for the digital elevation model (210) of the traffic route (10) by a data processing arrangement (50).

Images