Vehicle system and method for notifying current road conditions

Abstract

A system for notifying a road condition of a current vehicle may include a plurality of vehicle sensors, a vehicle communication system, and a controller in electrical communication with the plurality of vehicle sensors and the vehicle communication system. The controller is programmed to identify a measurement trigger. The controller is also programmed to perform measurements of the vehicle surroundings using the plurality of vehicle sensors in response to identifying the measurement trigger. The controller is further programmed to determine a measurement classification based at least in part on the measurement. The controller is also programmed to transmit the measurements and the measurement classifications to a remote server system using the vehicle communication system.

Description

Technical Field

[0001] The present invention relates to systems and methods for notifying authorities of road conditions.

[0002] To increase occupant awareness and convenience, the vehicle may be equipped with a perception and sensing system configured to monitor and measure various aspects of the vehicle's surroundings. The perception and sensing system may include, for example, a camera system, a LiDAR (light detection and ranging) system, and the like. The perception and sensing system may be used as part of a vehicle system to implement features such as, for example, advanced driver assistance systems (ADAS), autonomous driving systems (ADS), vehicle safety systems, and the like. However, current perception and sensing systems may not collect information about road conditions in the vehicle's surroundings and are unable to transmit information about road conditions to authorities. Therefore, manual information collection about road conditions may be required by authorities.

[0003] Therefore, while the perception and sensing systems and methods achieve their intended purposes, a new and improved system and method for notifying authorities of road conditions is needed. Summary of the invention

[0004] According to several aspects, a system for notifying authorities of road conditions is provided. The system may include a plurality of vehicle sensors, a vehicle communication system, and a controller in electrical communication with the plurality of vehicle sensors and the vehicle communication system. The controller is programmed to identify a measurement trigger. The controller is also programmed to perform a measurement of the vehicle's surroundings using the plurality of vehicle sensors in response to identifying the measurement trigger. The controller is also programmed to determine a measurement classification based at least in part on the measurement. The controller is also programmed to send the measurement and the measurement classification to a remote server system using the vehicle communication system.

[0005] In another aspect of the invention, the plurality of vehicle sensors includes at least a global navigation satellite system (GNSS). To identify a measurement trigger, the controller is further programmed to determine a location of the vehicle using the GNSS. To identify a measurement trigger, the controller is further programmed to compare the location of the vehicle to a predetermined geo-fenced location area. To identify a measurement trigger, the controller is further programmed to identify a measurement trigger in response to determining that the location of the vehicle is within the predetermined geo-fenced location area.

[0006] In another aspect of the invention, to identify the measurement trigger, the controller is further programmed to receive a trigger request from at least one of: an occupant of the vehicle and an authority. To identify the measurement trigger, the controller is further programmed to identify the measurement trigger based at least in part on the trigger request.

[0007] In another aspect of the invention, the plurality of vehicle sensors includes at least a camera system. To perform measurements of the vehicle surroundings, the controller is further programmed to capture one or more images of a remote vehicle using the camera system based at least in part on a trigger request. The trigger request includes at least one of: a remote vehicle license plate number, a remote vehicle model, and a remote vehicle color.

[0008] In another aspect of the present invention, to determine the measurement classification, the controller is further programmed to determine the measurement classification of the one or more images as a law enforcement measurement classification in response to determining that the remote vehicle in the one or more images includes at least one of: a remote vehicle license plate number, a remote vehicle model, and a remote vehicle color.

[0009] In another aspect of the present invention, in order to identify the measurement trigger, the controller is further programmed to perform a trigger measurement using a plurality of vehicle sensors. In order to identify the measurement trigger, the controller is further programmed to use a machine learning algorithm to identify the road condition in the trigger measurement. The road condition includes one of a normal road condition and an abnormal road condition. In order to identify the measurement trigger, the controller is further programmed to identify the measurement trigger in response to determining that the road condition is an abnormal road condition.

[0010] In another aspect of the invention, the plurality of vehicle sensors includes at least a camera system. To perform measurements of the vehicle's surroundings, the controller is further programmed to capture one or more images using the camera system. To perform measurements of the vehicle's surroundings, the controller is further programmed to store the one or more images in a non-transitory memory of the controller.

[0011] In another aspect of the invention, the plurality of vehicle sensors includes at least a global navigation satellite system (GNSS). To determine the measurement classification, the controller is further programmed to determine the location of the vehicle using the GNSS. To determine the measurement classification, the controller is further programmed to compare the location of the vehicle to a predetermined plurality of point of interest (POI) locations. To determine the measurement classification, the controller is further programmed to determine the measurement classification of the one or more images as a traveler measurement classification in response to determining that the location of the vehicle is within at least one predetermined distance from the plurality of POI locations.

[0012] In another aspect of the invention, to determine the measurement classification, the controller is further programmed to use a computer vision algorithm to identify a traffic sign in the one or more images. To determine the measurement classification, the controller is further programmed to use a computer vision algorithm to calculate a correlation value between the one or more images and a reference image. To determine the measurement classification, the controller is further programmed to determine the measurement classification of the one or more images as a damaged traffic sign measurement classification in response to identifying the traffic sign in the one or more images and in response to determining that the correlation value is less than a predetermined correlation threshold.

[0013] In another aspect of the invention, to determine the measurement classification, the controller is further programmed to use a machine learning algorithm to detect an adverse road condition in one or more images. The adverse road condition includes at least one of: a foreign object on the road, an animal on the road, and a damaged road surface. To determine the measurement classification, the controller is further programmed to determine a risk value for the adverse road condition. To determine the measurement classification, the controller is further programmed to determine the measurement classification of the one or more images as an adverse road condition measurement classification based at least in part on the adverse road condition and the risk value.

[0014] According to several aspects, a method for notifying an authority of a road condition is provided. The method may include performing a measurement of a vehicle's surroundings using a plurality of vehicle sensors. The method may also include determining a measurement classification based at least in part on the measurement. The method may also include transmitting the measurement and the measurement classification to an authority.

[0015] In another aspect of the invention, performing the measurements and determining the measurement classification may also include capturing one or more images of the vehicle's surroundings using a camera system. Performing the measurements and determining the measurement classification may also include detecting debris in the one or more images using a machine learning algorithm. Performing the measurements and determining the measurement classification may also include determining a risk value for the debris. Performing the measurements and determining the measurement classification may also include determining the measurement classification of the one or more images as a debris measurement classification based at least in part on the debris and the risk value.

[0016] In another aspect of the invention, performing measurements and determining measurement classifications may also include performing one or more measurements of the vehicle's surroundings using an infrared sensor configured to detect thermal radiation. Performing the measurements and determining the measurement classifications may also include detecting a fire in the vehicle's surroundings based at least in part on the one or more measurements. Performing measurements and determining measurement classifications may also include determining the measurement classification of the one or more measurements as a fire measurement classification in response to detecting a fire in the vehicle's surroundings.

[0017] In another aspect of the invention, performing the measurements and determining the measurement classification may also include capturing one or more images of the vehicle's surroundings using a camera system. Performing the measurements and determining the measurement classification may also include detecting graffiti in the one or more images using a machine learning algorithm. Performing the measurements and determining the measurement classification may also include determining the measurement classification of the one or more images as a graffiti measurement classification in response to detecting graffiti in the one or more images.

[0018] In another aspect of the invention, performing the measurements and determining the measurement classification may also include capturing one or more images of the vehicle's surroundings using a camera system. Performing the measurements and determining the measurement classification may also include detecting criminal activity in the one or more images using a machine learning algorithm. Performing the measurements and determining the measurement classification may also include determining the measurement classification of the one or more images as a criminal activity measurement classification in response to detecting criminal activity in the one or more images.

[0019] In another aspect of the invention, performing the measurements and determining the measurement classification may also include capturing one or more images of the vehicle's surroundings using a camera system. Performing the measurements and determining the measurement classification may also include detecting water on a road in the one or more images using a machine learning algorithm. Performing the measurements and determining the measurement classification may also include determining the measurement classification of the one or more images as a flood measurement classification in response to detecting water on the road in the one or more images.

[0020] In another aspect of the invention, performing the measurements and determining the measurement classification may also include capturing multiple images of the vehicle's surroundings using a camera system. Performing the measurements and determining the measurement classification may also include detecting one or more points of interest (POIs) in the multiple images using a machine learning algorithm. Performing the measurements and determining the measurement classification may also include determining the measurement classification of the multiple images as a traveler measurement classification in response to detecting the one or more POIs in the multiple images. Performing the measurements and determining the measurement classification may also include generating a film comprising the multiple images. Performing the measurements and determining the measurement classification may also include displaying the film to an occupant of the vehicle.

[0021] According to several aspects, a system for notifying authorities of road conditions is provided. The system may include a camera system. The system may also include a global navigation satellite system (GNSS). The system may also include a vehicle communication system. The system may also include a controller that is in electrical communication with the camera system, the GNSS, and the vehicle communication system. The controller is programmed to identify a measurement trigger. The controller is also programmed to capture one or more images of the vehicle's surroundings using the camera system in response to identifying the measurement trigger. The controller is also programmed to determine the location of each of the one or more images using the GNSS. The controller is also programmed to determine a measurement classification for each of the one or more images. The controller is also programmed to send the one or more images, the measurement classification for each of the one or more images, and the location of each of the one or more images to a remote server system using the vehicle communication system. The remote server system is configured to be accessed by the authorities.

[0022] In another aspect of the present invention, the measurement trigger comprises at least one of: a predetermined geo-fenced location area, a trigger request initiated by a vehicle occupant, and a trigger request sent by an authority.

[0023] In another aspect of the present invention, to determine the measurement classification, the controller is further programmed to identify road conditions in the one or more images. The road conditions include at least one of: points of interest (POI), damaged traffic signs, foreign objects on the road, animals on the road, damaged road surfaces, fires in the vehicle surroundings, graffiti, criminal activity, and water on the road. To determine the measurement classification, the controller is further programmed to determine the measurement classification based at least in part on the road conditions in the one or more images.

[0024] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a schematic diagram of a system for notifying authorities of a road condition of a vehicle according to an exemplary embodiment;

[0027] Figure 2 is a flow chart of a method for notifying authorities of a road condition of a vehicle according to an exemplary embodiment;

[0028] Figure 3 is a flow chart of a first exemplary embodiment of a method for identifying a measurement trigger according to an exemplary embodiment;

[0029] Figure 4 is a flow chart of a second exemplary embodiment of a method for identifying a measurement trigger according to an exemplary embodiment; and

[0030] Figure 5 is a flow chart of a third exemplary embodiment of a method for identifying a measurement trigger according to exemplary embodiments. DETAILED DESCRIPTION

[0031] The following description is merely exemplary in nature and is not intended to limit the present invention, application, or uses.

[0032] Various conditions encountered on or near roadways may require attention and / or resolution from civil authorities, including, for example, law enforcement agencies, fire departments, road maintenance agencies, etc. Civil authorities, however, may be tasked with servicing large and / or complex jurisdictions and may rely on manual investigation and / or reporting processes to identify issues requiring attention. Accordingly, the present invention provides a new and improved system and method for notifying authorities of road conditions.

[0033] refer to Figure 1 , a system for notifying authorities of road conditions is shown and is generally indicated by reference numeral 10. System 10 is shown with an exemplary vehicle 12. Although a passenger car is illustrated, it should be understood that vehicle 12 may be any type of vehicle without departing from the scope of the present invention. System 10 generally includes a controller 14, a plurality of vehicle sensors 16, and a vehicle communication system 18.

[0034] The controller 14 is used to implement a method 100 for notifying authorities of road conditions of a vehicle, as described below. The controller 14 includes at least one processor 20 and a non-transitory computer-readable storage device or medium 22. The processor 20 can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor of several processors associated with the controller 14, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or a device generally used to execute instructions. The computer-readable storage device or medium 22 can include volatile and non-volatile storage in, for example, a read-only memory (ROM), a random access memory (RAM), and a keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor 20 is powered off. The computer readable storage device or medium 22 may be implemented using a number of storage devices, such as a PROM (Programmable Read Only Memory), an EPROM (Electrical PROM), an EEPROM (Electrically Erasable PROM), flash memory, or another electrical, magnetic, optical, or combination storage device capable of storing data, some of which represents executable instructions that may be used by the controller 14 to control the various systems of the vehicle 12. The controller 14 may also include multiple controllers in electrical communication with each other. The controller 14 may be interconnected with additional systems and / or controllers of the vehicle 12, thereby allowing the controller 14 to access data such as the speed, acceleration, braking, and steering angle of the vehicle 12.

[0035] The controller 14 is in electrical communication with a plurality of vehicle sensors 16 and a vehicle communication system 18. In an exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a serial peripheral interface (SPI) network, etc. It should be understood that various additional wired and wireless technologies and communication protocols for communicating with the controller 14 are within the scope of the present invention.

[0036] The plurality of vehicle sensors 16 are used to acquire information related to the vehicle 12 . In the exemplary embodiment, the plurality of vehicle sensors 16 include at least a camera system 24 , a global navigation satellite system (GNSS) 26 , and an infrared sensor 28 .

[0037] In another exemplary embodiment, the plurality of vehicle sensors 16 further include sensors for determining performance data regarding the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 16 further include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor.

[0038] In another exemplary embodiment, the plurality of vehicle sensors 16 also include sensors for determining information about the environment within the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 16 also include at least one of a seat occupancy sensor, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, a cabin microphone, etc.

[0039] In another exemplary embodiment, the plurality of vehicle sensors 16 also include sensors for determining information about the environment surrounding the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 16 also include at least one of an ambient air temperature sensor and / or a barometric pressure sensor.

[0040] In another exemplary embodiment, at least one of the plurality of vehicle sensors 16 is a perception sensor capable of sensing objects and / or measuring distances in the environment surrounding the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 16 include a stereo camera with distance measurement capabilities. In one example, at least one of the plurality of vehicle sensors 16 is fixed to the interior of the vehicle 12, for example, in the roof of the vehicle 12, with a view through the windshield 96 of the vehicle 12. In another example, at least one of the plurality of vehicle sensors 16 is fixed to the exterior of the vehicle 12, for example, on the roof of the vehicle 12, with a view of the environment 94 surrounding the vehicle 12. It should be understood that various additional types of perception sensors, such as, for example, LiDAR sensors, ultrasonic ranging sensors, radar sensors, and / or time-of-flight sensors, are within the scope of the present disclosure. As described above, the plurality of vehicle sensors 16 are electrically connected to the controller 14.

[0041] The camera system 24 is a perception sensor for capturing images and / or video of the environment surrounding the vehicle 12. In an exemplary embodiment, the camera system 24 includes a photo and / or video camera positioned to observe the environment surrounding the vehicle 12. In a non-limiting example, the camera system 24 includes a camera fixed to the interior of the vehicle 12, such as in the roof of the vehicle 12, with a view through the windshield. In another non-limiting example, the camera system 24 includes a camera fixed to the exterior of the vehicle 12, such as on the roof of the vehicle 12, with a view of the environment in front of the vehicle 12.

[0042] In another exemplary embodiment, the camera system 24 is a surround view camera system that includes multiple cameras (also referred to as satellite cameras) that are arranged to provide a view of the environment adjacent to all sides of the vehicle 12. In a non-limiting example, the camera system 24 includes a forward facing camera (e.g., mounted in the front grille of the vehicle 12), a rearward facing camera (e.g., mounted on the rear tailgate of the vehicle 12), and two opposing cameras (e.g., mounted below each of the two side view mirrors of the vehicle 12). In another non-limiting example, the camera system 24 also includes an additional rear view camera mounted near the center high mounted brake light of the vehicle 12.

[0043] It should be understood that camera systems with additional cameras and / or additional mounting locations are within the scope of the present invention. It should also be understood that cameras with various sensor types, including, for example, a charge coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor, and / or a high dynamic range (HDR) sensor, are within the scope of the present invention. In addition, cameras with various lens types, including, for example, a wide angle lens and / or a narrow angle lens, are also within the scope of the present invention.

[0044] GNSS26 is used to determine the geographic location of vehicle 12. In an exemplary embodiment, GNSS26 is a global positioning system (GPS). In a non-limiting example, GPS includes a GPS receiver antenna (not shown) and a GPS controller (not shown) in electrical communication with the GPS receiver antenna. The GNSS receiver antenna receives signals from a plurality of satellites, and the GNSS controller calculates the geographic location of the vehicle based on the signals received by the GNSS receiver antenna. In an exemplary embodiment, GNSS26 also includes a map. The map includes information about infrastructure such as municipality boundaries, roads, railways, sidewalks, buildings, etc. Therefore, the geographic location of vehicle 12 is contextualized using map information. In a non-limiting example, a map is retrieved from a remote source using a wireless connection. In another non-limiting example, the map is stored in a database of GNSS26. It should be understood that various additional types of satellite-based radio navigation systems, such as, for example, the Global Positioning System (GPS), Galileo, GLONASS, and the Beidou Navigation Satellite System (BDS) are within the scope of the present invention. It should be understood that the GNSS 26 may be integrated with the controller 14 (eg, on the same circuit board as the controller 14 , or on the same circuit board as the controller 14 , or on the same circuit board as a portion of the controller 14 ) without departing from the scope of the present invention.

[0045] The infrared sensor 28 is used to detect thermal radiation in the environment surrounding the vehicle 12. In an exemplary embodiment, the infrared sensor 28 determines the temperature of objects in the environment surrounding the vehicle 12 by measuring the thermal radiation emitted by the objects in the environment surrounding the vehicle 12. In a non-limiting example, the infrared sensor 28 includes an infrared sensor element and a signal processing unit. In some embodiments, the infrared sensor 28 may also include one or more lenses to focus the infrared radiation onto the infrared sensor element. The infrared sensor element detects the infrared radiation and converts the infrared radiation into an electrical signal. The electrical signal is then sent to the signal processing unit. The signal processing unit interprets the data and calculates the temperature of the objects in the environment surrounding the vehicle 12 based on the intensity of the detected infrared radiation. It should be understood that additional devices operable for non-contact temperature measurement of objects in the environment surrounding the vehicle 12 are within the scope of the present invention.

[0046] The vehicle communication system 18 is used by the controller 14 to communicate with other systems external to the vehicle 12. For example, the vehicle communication system 18 includes the ability to communicate with vehicle communication ("V2V" communication), infrastructure communication ("V2I" communication), remote systems at a remote call center (e.g., ON-STAR by GENERAL MOTORS), and / or personal devices. In general, the term vehicle-to-everything communication ("V2X" communication) refers to communication between the vehicle 12 and any remote system (e.g., vehicle, infrastructure, and / or remote system). In some embodiments, the vehicle communication system 18 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communication (e.g., using GSMA standards such as, for example, SGP.02, SGP.22, SGP.32, etc.). Therefore, the vehicle communication system 18 may also include an embedded universal integrated circuit card (eUICC) configured to store at least one cellular connectivity configuration profile, such as an embedded subscriber identity module (eSIM) profile. The vehicle communication system 18 is also configured to communicate via a personal area network (e.g., Bluetooth), near field communication (NFC), and / or any additional type of radio frequency communication. However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels and / or mobile telecommunication protocols based on the third generation partnership project (3GPP) standard are also considered to be within the scope of the present invention. DSRC channels refer to unidirectional or bidirectional short-range to medium-range wireless communication channels specific to automotive use and corresponding protocols and standards sets. 3GPP refers to a partnership between several standard organizations that develop protocols and standards for mobile telecommunications. 3GPP standards are structured as "releases." Therefore, communication methods based on 3GPP releases 14, 15, 16 and / or future 3GPP releases are considered to be within the scope of the present invention. Therefore, the vehicle communication system 18 may include one or more antennas and / or transceivers for receiving and / or transmitting signals, such as collaborative sensing messages (CSM). The vehicle communication system 18 is configured to wirelessly transmit information between the vehicle 12 and another vehicle. In addition, the vehicle communication system 18 is configured to wirelessly transmit information between the vehicle 12 and infrastructure or other vehicles. It should be understood that the vehicle communication system 18 may be integrated with the controller 14 (e.g., integrated on the same circuit board as the controller 14, integrated on the same circuit board as the controller 14, or integrated on the same circuit board as a portion of the controller 14) without departing from the scope of the present invention. As described above, the vehicle communication system 18 is in electrical communication with the controller 14.

[0047] Continue to refer Figure 1, a remote server system is shown, generally indicated by reference numeral 30. The remote server system 30 includes a server controller 32 in electrical communication with a server database 34 and a server communication system 36. In a non-limiting example, the remote server system 30 is located in a server farm, data center, etc., and is connected to the Internet.

[0048] The server controller 32 includes at least one server processor 38 and a server non-transitory computer-readable storage device or server media 40. The description of types and configurations given above for the controller 14 also applies to the server controller 32. In some examples, the server controller 32 may differ from the controller 14 in that the server controller 32 is capable of a higher processing speed, includes more memory, includes more input / output, etc. In a non-limiting example, the server processor 38 and server media 40 of the server controller 32 are similar in structure and / or function to the processor 20 and media 22 of the controller 14, as described above.

[0049] The server database 34 is used to store measurements and measurement classifications, which will be discussed in more detail below. In an exemplary embodiment, the server database 34 includes one or more mass storage devices, such as, for example, a hard disk drive, a tape drive, a magneto-optical drive, an optical disk, a solid-state drive, and / or additional devices that can be used to store data in a persistent and machine-readable manner. In some examples, the one or more mass storage devices can be configured to provide redundancy in the event of hardware failure and / or data corruption using, for example, a redundant array of independent disks (RAID). In a non-limiting example, the server controller 32 can execute software, such as, for example, a database management system (DBMS), that allows organization and access to data stored on one or more mass storage devices.

[0050] The server communication system 36 is used to communicate with external systems (such as, for example, the controller 14) via the vehicle communication system 18. In a non-limiting example, the server communication system 36 is similar in structure and / or function to the vehicle communication system 18 described above. In some examples, the server communication system 36 may differ from the vehicle communication system 18 in that the server communication system 36 is capable of higher power signal transmissions, more sensitive signal reception, higher bandwidth transmissions, additional transmission / reception protocols, etc.

[0051] refer to Figure 2, a flow chart of a method 100 for notifying an official vehicle of a road condition is shown. The method 100 begins at box 102 and proceeds to box 104. At box 104, the controller 14 identifies a measurement trigger. Within the scope of the present invention, a measurement trigger is an event that prompts the controller 14 to perform a measurement using a plurality of vehicle sensors 16, which will be discussed in more detail below. The identification of a measurement trigger will be discussed in more detail below. After box 104, the method 100 proceeds to box 106.

[0052] refer to Figure 3 , a first exemplary embodiment 104a of box 104 is shown. The first exemplary embodiment 104a begins at box 302. At box 302, the controller 14 determines the location of the vehicle 12 using the GNSS 26. After box 302, the first exemplary embodiment 104a proceeds to box 304. At box 304, the location of the vehicle 12 determined at box 302 is compared with a predetermined geo-fenced location area. Within the scope of the present invention, a predetermined geo-fenced location area is a geographical area that has been predetermined as an area of ​​interest for performing measurements. In a non-limiting example, the predetermined geo-fenced location area includes one or more road segments, neighborhoods, city blocks, etc. In an exemplary embodiment, the predetermined geo-fenced location can be updated by the remote server system 30 and received by the controller 14 using the vehicle communication system 18, using, for example, a system learning and update (SLU) data collection system. If the location of the vehicle 12 determined at box 302 is not within the predetermined geo-fenced location area, the first exemplary embodiment 104a proceeds to box 306. If the location of the vehicle 12 determined at block 302 is within the predetermined geo-fenced location area, the first exemplary embodiment 104 a proceeds to block 308 .

[0053] At block 306, a measurement trigger is not identified. After block 306, the first exemplary embodiment 104a ends, and the method 100 proceeds as described below. At block 308, a measurement trigger is identified. After block 308, the first exemplary embodiment 104a ends, and the method 100 proceeds as described below.

[0054] refer to Figure 4, a second exemplary embodiment 104b of box 104 is shown. The second exemplary embodiment 104b starts at box 402. At box 402, the controller 14 receives a trigger request. Within the scope of the present invention, a trigger request is a request for the system 10 to perform a measurement in response to determining that one or more conditions have been met. The trigger request contains one or more conditions. In an exemplary embodiment, the trigger request is received from an occupant of the vehicle 12. In a non-limiting example, the occupant of the vehicle 12 uses a human-machine interface (HMI) to input the trigger request. For example, the trigger request may include a request from the occupant to capture an image of a specific point of interest (POI). Within the scope of the present invention, POIs include, for example, businesses, restaurants, vehicle service centers, hospitals, police stations, gas stations, vehicle charging stations, entertainment areas, nature reserves, museums, theaters, tourist attractions, statues, monuments, memorials, etc. In another exemplary embodiment, the trigger request is received from an authority. Within the scope of the present invention, an authority includes an organization and / or individual responsible for maintaining public road infrastructure, including, for example, a law enforcement agency, a fire department, a road maintenance agency, etc. In a non-limiting example, the authority provides the trigger request to the remote server system 30. After block 402 , the second exemplary embodiment 104 b proceeds to block 404 .

[0055] At block 404, the controller 14 uses the plurality of vehicle sensors 16 to perform measurements of the environment surrounding the vehicle 12. In the exemplary embodiment, the controller 14 uses the camera system 24 to capture images of the environment surrounding the vehicle 12. After block 404, the second exemplary embodiment 104b proceeds to block 406.

[0056] At box 406, the measurement performed at box 404 is compared to the trigger request received at box 402. For example, a law enforcement agency may provide a trigger request that includes a remote vehicle license plate number, a remote vehicle model, and / or a remote vehicle color. If the measurement performed at box 404 (i.e., the image captured at box 404) includes a remote vehicle license plate number, a remote vehicle model, and / or a remote vehicle color, then it is determined that the measurement matches the trigger request. In a non-limiting example, determination of the remote vehicle license plate number is performed according to the methods discussed in Lubna et al., "Automatic License Plate Recognition: A Detailed Review of Related Algorithms" (Sensors, 21, 3028, April 2021), the entire contents of which are incorporated herein by reference.

[0057] In another example, a road maintenance agency may provide a trigger request that includes all types of traffic signs. Thus, if the measurement performed at block 404 (i.e., the image captured at block 404) includes a traffic sign, then the measurement is determined to match the trigger request. If the measurement is not determined to match the trigger request, the second exemplary embodiment 104b returns to block 404 to capture a new measurement. If it is determined that the measurement matches the trigger request, the second exemplary embodiment 104b proceeds to block 408.

[0058] At block 408, a measurement trigger is identified. After block 408, the second exemplary embodiment 104b ends and the method 100 proceeds as described below.

[0059] refer to Figure 5 , a third exemplary embodiment 104c of the block 104 is shown. The third exemplary embodiment 104c begins at block 502. At block 502, the controller 14 uses the plurality of vehicle sensors 16 to perform a triggered measurement of the environment surrounding the vehicle 12. Within the scope of the present invention, a triggered measurement is a measurement used to identify a measurement trigger. In an exemplary embodiment, the controller 14 uses the camera system 24 to capture an image of the environment surrounding the vehicle 12. After block 502, the third exemplary embodiment 104c proceeds to block 504.

[0060] At box 504, the controller 14 uses a machine learning algorithm to evaluate the trigger measurement. In an exemplary embodiment, the machine learning algorithm is configured to identify road conditions in the trigger measurement. Within the scope of the present invention, road conditions include normal road conditions and abnormal road conditions. Within the scope of the present invention, abnormal road conditions include, for example, foreign objects on the road, animals on the road, water on the road, damaged road surfaces, damaged traffic signs, fires, criminal activities, and / or graffiti.

[0061] In a non-limiting example, the machine learning algorithm includes multiple layers, including an input layer and an output layer, and one or more hidden layers. The input layer receives the trigger measurement as input. The input is then passed to the hidden layer. Each hidden layer applies a transformation (e.g., a nonlinear transformation) to the data and passes the result to the next hidden layer until the final hidden layer. The output layer produces the road condition.

[0062] To train a machine learning algorithm, an input dataset and its corresponding road conditions are used. The algorithm is trained by adjusting the internal weights between nodes in each hidden layer to minimize the prediction error. During training, optimization techniques (e.g., gradient descent) are used to adjust the internal weights to reduce the prediction error. The training process is repeated for the entire dataset until the prediction error is minimized, and the resulting trained model is then used to classify new input data.

[0063] After the machine learning algorithm is fully trained, the algorithm is able to accurately and precisely determine road conditions based on the trigger measurements. By adjusting the weights between nodes in each hidden layer during training, the algorithm "learns" to recognize patterns in the data that indicate road conditions. After block 504, the third exemplary embodiment 104c proceeds to block 506.

[0064] At block 506, if an abnormal road condition is not identified at block 504, the third exemplary embodiment 104c returns to block 502 to perform another measurement. If an abnormal road condition is identified at block 504, the third exemplary embodiment 104c proceeds to block 508. At block 508, a measurement trigger is identified. After block 508, the third exemplary embodiment 104c ends and the method 100 proceeds as described below.

[0065] Reference again Figure 2 At block 106 , if a measurement trigger is not identified at block 104 , the method 100 enters a standby state at block 108 . If a measurement trigger is identified at block 104 , the method 100 proceeds to block 110 .

[0066] At box 110, the controller 14 uses the plurality of vehicle sensors 16 to perform measurements of the environment around the vehicle 12. In an exemplary embodiment, performing the measurements includes capturing one or more images of the environment around the vehicle 12 using the camera system 24. In a non-limiting example, the one or more images are stored in the medium 22 of the controller 14 for later retrieval. In another exemplary embodiment, performing the measurements also includes determining a location of the vehicle 12 corresponding to each of the one or more images using the GNSS 26. The location of the vehicle 12 is stored in the medium 22 of the controller 14 along with the one or more images. In another exemplary embodiment, performing the measurements also includes performing one or more measurements using the infrared sensor 28. After box 110, the method 100 proceeds to box 112.

[0067] At box 112, the controller 14 extracts importance information from the one or more images captured at box 110. Within the scope of the present invention, importance information includes adverse road conditions (i.e., foreign objects on the road, animals on the road, water on the road, and / or damaged road surfaces), damaged traffic signs, fires, criminal activities, graffiti, and / or points of interest (POIs). In an exemplary embodiment, the controller 14 extracts importance information from the one or more images captured at box 110 using a computer vision algorithm. In a non-limiting example, the computer vision algorithm uses machine learning techniques to analyze pixel-level information of an input image to detect and classify objects or patterns of interest. In a non-limiting example, the computer vision algorithm begins by preprocessing the input image to reduce noise through techniques such as, for example, image resizing, normalization, and / or filtering. Subsequently, the computer vision algorithm extracts relevant features from the input image using methods such as, for example, edge detection, corner detection, texture analysis, and the like. The computer vision algorithm can then use a machine learning model, such as, for example, a convolutional neural network (CNN), to classify and label relevant features of the input image based on learned patterns and associations. After frame 112 , the method 100 proceeds to frame 114 .

[0068] At block 114, the controller 14 determines a measurement classification for the environmental measurement performed at block 110. In an exemplary embodiment, the measurement classification is determined based at least in part on the trigger request. For example, if the environmental measurement performed at block 110 corresponds to a trigger request received from a law enforcement agency, as described above with reference to Figure 4 As discussed above, the measurement classification is determined to be a law enforcement measurement classification. In another example, if the environmental measurement performed at block 110 corresponds to a trigger request received from an occupant, as described above with reference to Figure 4 As discussed above, the measurement classification is determined to be the classification requested by the occupant. In another example, if the environmental measurement performed at box 110 corresponds to a trigger request received from a road maintenance agency, as described above with reference to Figure 4 As discussed, the measurement category is determined to be the road maintenance measurement category.

[0069] In another exemplary embodiment, the measurement classification is determined based at least in part on the content of the environmental measurement performed at box 110. In a non-limiting example, the environmental measurement performed at box 110 is analyzed using a computer vision algorithm to determine whether the environmental measurement performed at box 110 contains a traffic sign. If the environmental measurement performed at box 110 contains a traffic sign, the controller 14 then uses the computer vision algorithm to calculate a correlation value between the environmental measurement performed at box 110 (i.e., one or more images captured by the camera system 24) and a reference image of a complete traffic sign. Within the scope of the present invention, the correlation value quantifies the level of similarity between the environmental measurement performed at box 110 and the reference image. The controller 14 then compares the correlation value with a predetermined correlation threshold (e.g., 60%). If the correlation value is less than the predetermined correlation threshold, it is determined that the traffic sign in the environmental measurement performed at box 110 includes a damaged traffic sign. Therefore, the measurement classification is determined to be a damaged traffic sign measurement classification.

[0070] In another non-limiting example, damaged traffic signs are detected according to the method discussed in "Destruction Detection of Traffic Signs by Using Position Histogram Matching" by Yang et al., Journal of the Korean Multimedia Society, Vol. 15, No. 3 (pp. 312-322), March 2012, the entire contents of which are incorporated herein by reference.

[0071] In another non-limiting example, a machine learning algorithm is used to analyze the environmental measurements performed at box 110 to determine whether the environmental measurements performed at box 110 include an adverse road condition. Within the scope of the present invention, the adverse road condition includes at least one of: a foreign object on the road, an animal on the road, water on the road, and / or a damaged road surface. If the environmental measurements performed at box 110 include an adverse road condition, the controller 14 then determines a risk value for the adverse road condition. Within the scope of the present invention, the risk value quantifies the level of risk that the adverse road condition poses to a vehicle on the road.

[0072] In an exemplary embodiment, to determine the risk value, the controller 14 uses a machine learning algorithm configured to determine the risk value based on the adverse road condition. In an exemplary embodiment, the risk value is determined based on the type and severity of the adverse road condition. In a non-limiting example, if the adverse road condition includes a complete blockage of the road, the risk value is relatively higher than if the adverse road condition includes a minimal blockage of the road.

[0073] The controller 14 then compares the risk value to a predetermined risk value threshold. If the risk value is higher than the predetermined risk value threshold, the measurement classification is determined to be an adverse road condition measurement classification.

[0074] It should be understood that the measurement classification can be further specified based on the type of adverse road conditions. For example, if the environmental measurement performed at box 110 includes debris, such as, for example, garbage, the measurement classification can be determined as a debris measurement classification. In a non-limiting example, according to the method discussed in "A Framework for Roadside Trash Identification and Facial Recognition Using Convolutional Neural Networks" by Abiga Sansuri et al. (Proceedings of the ACM / CSI / IEEECS Research and Industry Symposium on IoT Cloud for Social Applications (IoTCloud'21), the entire contents of which are incorporated herein by reference.

[0075] If the environmental measurements performed at box 110 include animals on the road, the measurement classification can be determined as a roadkill measurement classification. If the environmental measurements performed at box 110 include water on the road, the measurement classification can be determined as a flood measurement classification. In a non-limiting example, water on the road is detected according to the methods discussed in Santana et al., "Water Detection with Segment-Guided Dynamic Texture Recognition," 2012, IEEE International Conference on Robotics and Bionics, ROBIO 2012 - Conference Abstracts, December 2012, the entire contents of which are incorporated herein by reference.

[0076] If the environmental measurements performed at block 110 include a damaged road surface (e.g., potholes), the measurement classification may be determined as a damaged road surface measurement classification. It should be appreciated that the debris measurement classification, road damage measurement classification, flood measurement classification, and damaged road surface measurement classification are determined in a similar manner to the adverse road condition measurement classification described above.

[0077] In another non-limiting example, a machine learning algorithm is used to analyze the environmental measurements performed at box 110 to determine whether the environmental measurements performed at box 110 indicate a fire in the environment surrounding the vehicle 12. In a non-limiting example, if the environmental measurements performed at box 110 include thermal radiation measurements performed using the infrared sensor 28, the thermal radiation measurements are compared to a predetermined thermal radiation threshold. If one or more thermal radiation measurements are greater than or equal to the thermal radiation threshold, the measurement classification is determined to be a fire measurement classification. In another non-limiting example, if the environmental measurements performed at box 110 include one or more images captured by the camera system 24, a machine learning algorithm is used to detect a fire in the one or more images. In another non-limiting example, in the environmental measurements performed at box 110, a fire is detected using the method discussed in "A Review of Early Forest Fire Detection Systems Using Optical Remote Sensing" by Barmpoutis et al. (Sensors, 20, 6442, November 2020), the entire contents of which are incorporated herein by reference.

[0078] In another non-limiting example, a machine learning algorithm is used to analyze the environmental measurements performed at box 110 to determine whether the environmental measurements performed at box 110 include graffiti. In a non-limiting example, if the environmental measurements performed at box 110 include one or more images captured by the camera system 24, a machine learning algorithm is used to detect graffiti in the one or more images. In another non-limiting example, graffiti is detected in the environmental measurements performed at box 110 using the method discussed in "Deep Learning-Based Graffiti Detection: A Study Using Images from Lisbon Streets" by Foga Acai et al. (Applied Science, 13, 2249, February 2023), the entire contents of which are incorporated herein by reference. If the environmental measurements performed at box 110 are determined to include graffiti, the measurement classification is determined to be a graffiti measurement classification.

[0079] In another non-limiting example, the measurement of the environment performed at block 110 is analyzed using a machine learning algorithm to determine whether the measurement of the environment performed at block 110 includes criminal activity (e.g., theft, vandalism, violence, etc.). In a non-limiting example, if the measurement of the environment performed at block 110 includes one or more images captured by the camera system 24, the machine learning algorithm is used to detect criminal activity in the one or more images. If the measurement of the environment performed at block 110 is determined to include criminal activity, the measurement classification is determined to be a criminal activity measurement classification.

[0080] In another exemplary embodiment, the measurement classification is determined based at least in part on the location of the environmental measurement performed at box 110 determined using GNSS 26. In a non-limiting example, if the location of the environmental measurement performed at box 110 is within a predetermined distance threshold (e.g., ten meters) from a POI (e.g., a museum, a statue, a monument, a memorial, a nature reserve, a tourist attraction, etc.), the measurement classification is determined to be a traveler measurement classification. In another non-limiting example, if the location of the environmental measurement performed at box 110 is within a predetermined distance of one of a predetermined plurality of POI locations (e.g., stored in the media 22 of the controller 14), the measurement classification is determined to be a traveler measurement classification. In a non-limiting example, a machine learning algorithm is used to analyze the environmental measurement performed at box 110 (e.g., one or more images captured by the camera system 24) to detect one or more points of interest (POI) in the environmental measurement performed at box 110. If one or more POIs are detected in the environmental measurement performed at box 110, the measurement classification is determined to be a traveler measurement classification.

[0081] It will be appreciated that additional measurement categories and subcategories for providing information to authorities and / or for increasing occupant comfort and convenience are within the scope of the present invention. Following frame 114 , the method 100 proceeds to frame 116 .

[0082] At box 116, the controller 14 uses the vehicle communication system 18 to send the environmental measurements performed at box 110 (e.g., one or more images captured using the camera system 24), the location of the environmental measurements performed at box 110, and the measurement classification determined at box 114 to the remote server system 30. In an exemplary embodiment, the server controller 32 of the remote server system 30 uses the server communication system 36 to receive the environmental measurements performed at box 110, the location of the environmental measurements performed at box 110, and the measurement classification determined at box 114. The server controller 32 then stores the environmental measurements performed at box 110, the location of the environmental measurements performed at box 110, and the measurement classification determined at box 114 in the server database 34 for later retrieval.

[0083] In an exemplary embodiment, a cellular data connection is used to send the environmental measurements performed at block 110, the location of the environmental measurements performed at block 110, and the measurement classification determined at block 114. In another exemplary embodiment, a WiFi connection is used to send the environmental measurements performed at block 110, the location of the environmental measurements performed at block 110, and the measurement classification determined at block 114.

[0084] In another exemplary embodiment, the controller 14 compiles all measurements (e.g., images and / or videos) having a visitor measurement classification into a multimedia presentation (e.g., a photo collage, a movie, a slide show, etc.) and displays the multimedia presentation to the occupants using, for example, the HMI of the vehicle 12. After frame 116, the method 100 proceeds to frame 118.

[0085] At box 118, the server controller 32 provides the environmental measurements performed at box 110, the locations of the environmental measurements performed at box 110, and the measurement classifications determined at box 114 and stored in the server database 34 at box 116 to the authorities. In an exemplary embodiment, the authorities can use the server communication system 36 to establish a connection with the server controller 32 and request measurements with a specific measurement classification and / or a specific location. In another exemplary embodiment, the server controller 32 uses the server communication system 36 to automatically send measurements with a specific measurement classification to a specific agency. In a non-limiting example, measurements with a law enforcement measurement classification, a graffiti measurement classification, and / or a criminal activity measurement classification are transmitted to the law enforcement agency.

[0086] In another non-limiting example, measurements with a road maintenance measurement classification, a damaged traffic sign measurement classification, an adverse road condition measurement classification, a debris measurement classification, a road damage measurement classification, a flood measurement classification, and / or a damaged road surface measurement classification are sent to a road maintenance agency. In another non-limiting example, measurements with a fire measurement classification are transmitted to a fire department. After block 118, method 100 enters a standby state at block 108.

[0087] In another non-limiting example, the server controller 32 uses the server communication system 36 to provide an application programming interface (API) that allows authorities to send trigger requests and receive information stored in the server database 34 .

[0088] In an exemplary embodiment, the controller 14 repeatedly exits the standby state 108 and restarts the method 100 at block 102. In a non-limiting example, the controller 14 exits the standby state 108 and restarts the method 100 on a timer, for example, every three hundred milliseconds.

[0089] The system 10 and method 100 of the present invention have several advantages. By collecting and aggregating road condition data, authorities can be quickly notified of problems that need to be addressed. Using the method 100, the system 10 can utilize additional computing power to provide information to authorities. In addition, by recording images and / or videos of POIs, the system 10 and method 100 can be used to increase passenger comfort and convenience.

[0090] The description of the present invention is merely exemplary in nature, and variations that do not depart from the gist of the present invention are intended to be within the scope of the present invention. Such variations are not to be regarded as a departure from the spirit and scope of the present invention.

Claims

1. A system for notifying authorities of road conditions for a vehicle, the system comprising: Multiple vehicle sensors; Vehicle communication systems; as well as a controller in electrical communication with the plurality of vehicle sensors and the vehicle communication system, wherein the controller is programmed to: Identify measurement triggers; In response to identifying the measurement trigger, performing a measurement of an environment surrounding the vehicle using the plurality of vehicle sensors; determining a measurement classification based at least in part on the measurement; and The measurements and the measurement classifications are transmitted to a remote server system using the vehicle communication system.

2. The system according to claim 1, wherein: The plurality of vehicle sensors includes at least a Global Navigation Satellite System (GNSS), and wherein, to identify the measurement trigger, the controller is further programmed to: determining a position of the vehicle using the GNSS; comparing the location of the vehicle to a predetermined geo-fenced location area; and The measurement trigger is identified in response to determining that the location of the vehicle is within the predetermined geo-fenced location area.

3. The system according to claim 1, wherein: In order to identify the measurement trigger, the controller is also programmed to: receiving a trigger request from at least one of: an occupant of the vehicle and an authority; and The measurement trigger is identified based at least in part on the trigger request.

4. The system according to claim 3, wherein: The plurality of vehicle sensors includes at least one camera system, and wherein, in order to perform said measuring of the environment surrounding the vehicle, the controller is further programmed to: One or more images of a remote vehicle are captured using the camera system based at least in part on the trigger request, wherein the trigger request includes at least one of: a remote vehicle license plate number, a remote vehicle model, and a remote vehicle color.

5. The system according to claim 4, wherein: To determine the measurement classification, the controller is also programmed to: The measurement classification of the one or more images is determined to be a law enforcement measurement classification in response to determining that the remote vehicle in the one or more images includes at least one of: the remote vehicle license plate number, the remote vehicle model, and the remote vehicle color.

6. The system according to claim 1, wherein: In order to identify the measurement trigger, the controller is also programmed to: performing a triggered measurement using the plurality of vehicle sensors; Using a machine learning algorithm to identify a road condition in the trigger measurement, wherein the road condition includes one of: a normal road condition and an abnormal road condition; and The measurement trigger is identified in response to determining that the road condition is the abnormal road condition.

7. The system according to claim 1, wherein: The plurality of vehicle sensors includes at least one camera system, and wherein, in order to perform said measuring of the environment surrounding the vehicle, the controller is further programmed to: capturing one or more images using the camera system; and The one or more images are stored in a non-transitory memory of the controller.

8. The system according to claim 7, wherein: The plurality of vehicle sensors includes at least a global navigation satellite system (GNSS), and wherein, to determine the measurement classification, the controller is further programmed to: determining a position of the vehicle using the GNSS; comparing the position of the vehicle to a predetermined plurality of point of interest (POI) locations; and The measurement classification of the one or more images is determined as a traveler measurement classification in response to determining that the location of the vehicle is within at least one predetermined distance from the plurality of POI locations.

9. The system according to claim 7, wherein: To determine the measurement classification, the controller is also programmed to: recognizing a traffic sign in the one or more images using a computer vision algorithm; calculating a correlation value between the one or more images and a reference image using the computer vision algorithm; as well as In response to identifying the traffic sign in the one or more images and in response to determining that the correlation value is less than the predetermined correlation threshold, the measurement classification of the one or more images is determined to be a damaged traffic sign measurement classification.

10. The system according to claim 7, wherein: To determine the measurement classification, the controller is also programmed to: detecting adverse road conditions in the one or more images using a machine learning algorithm, wherein the adverse road condition comprises at least one of: a foreign object on the road, an animal on the road, and a damaged road surface; determining a risk value of the adverse road condition; and The measured classification of the one or more images is determined as the adverse road condition measured classification based at least in part on the adverse road condition and the risk value.

Images