Method, apparatus and computer program product for determination of ground target characteristics

Abstract

The present disclosure provides a method, an apparatus, and a computer program product for determination of ground target characteristics. Determination of a ground target characteristic based on a vehicle record is carried out by detecting a vehicle response of the vehicle, where the vehicle response is triggered by a read threshold of at least one trigger sensor of the vehicle; recording outputs of at least one trigger sensor and at least one associated sensor of the vehicle in response to the detection of the vehicle reaction; and determining a target characteristic of the ground based on the recorded output associated with the at least one trigger sensor, the recorded output associated with the at least one associated sensor, and the at least one characteristic of the vehicle.

Description

Technical Field

[0001] The present disclosure relates to methods, apparatus, and computer program products for vehicle-based determination of ground target characteristics. Background Art

[0002] Onboard sensors are becoming more common. With the advent of autonomous driving, not only are conventional cameras being added to vehicles, but arrays of sensors such as LiDAR (laser radar) are also being added to vehicles to supplement existing sensors such as GPS (Global Positioning System). Through the combination of these sensors, the vehicle can obtain a detailed representation of the world around the vehicle. By connecting to the Internet, the vehicle can communicate with aggregation servers. Most of the sensor data that makes up this representation is overwritten due to storage issues, but a portion of the information is stored for the purpose of using real-time executable information or historical information to create simulation environments for training and prediction of ADAS (Advanced Driver Assistance Systems), maps, etc. The usefulness of this information increases as its quality increases. Generally, as the quality of the information increases, the requested data transmission and storage capacity also increases. Summary of the invention

[0003] The present disclosure provides methods, apparatus, and computer program products for vehicle-based determination of ground target characteristics.

[0004] The method of the present disclosure includes: detecting a vehicle reaction of a vehicle, wherein the vehicle reaction is triggered by a reading threshold of at least one trigger sensor of the vehicle; recording outputs of at least one trigger sensor and at least one associated sensor of the vehicle in response to the detection of the vehicle reaction; and determining a target characteristic of the ground based on the recorded output associated with the at least one trigger sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle.

[0005] The computer program product of the present disclosure is a computer program product having recorded thereon instructions that can be executed by one or more processors to perform actions, the actions including: detecting a vehicle reaction of a vehicle, wherein the vehicle reaction is triggered by a reading threshold of at least one trigger sensor of the vehicle; recording outputs of at least one trigger sensor and at least one associated sensor of the vehicle in response to the detection of the vehicle reaction; and determining a target characteristic of the ground based on the recorded output associated with at least one trigger sensor, the recorded output associated with at least one associated sensor, and at least one characteristic of the vehicle.

[0006] The apparatus of the present disclosure is provided with a controller including a circuit, wherein the circuit is configured to: detect a vehicle reaction of a vehicle, wherein the vehicle reaction is triggered by a reading threshold of at least one trigger sensor of the vehicle; record outputs of at least one trigger sensor and at least one associated sensor of the vehicle in response to the detection of the vehicle reaction; and determine a target characteristic of the ground based on the recorded output associated with the at least one trigger sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle.

[0007] According to the present disclosure, methods, apparatus, and computer program products can be provided for determining ground object characteristics based on vehicle records. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The scheme of the present disclosure can be best understood by using the accompanying drawings and reading the detailed description below. It should be noted that, in accordance with standard practice in the industry, various features are not depicted to scale. In fact, in order to make the description clear, the size of various features can be increased or decreased arbitrarily.

[0009] Figure 1 A schematic diagram of a system related to the ingestion of query content over a mobile computing network in accordance with at least some embodiments of the subject disclosure.

[0010] Figure 2 A diagram of a graphical user interface (GUI) related to querying content of a mobile computing network according to at least some embodiments of the subject disclosure.

[0011] Figure 3 is a diagram of a data structure for a content acquisition task of at least some embodiments of the subject disclosure.

[0012] Figure 4 A schematic diagram of a query content acquisition system according to at least a portion of the disclosed embodiments of the subject matter.

[0013] Figure 5 A flow of actions related to determining a ground objective characteristic based on vehicle records is provided for at least some embodiments of the subject disclosure.

[0014] Figure 6 The present invention is a flow chart of detecting vehicle reactions in at least some embodiments of the subject disclosure.

[0015] Figure 7 The present invention is a flow of actions for recording vehicle responses in accordance with at least some embodiments of the subject disclosure.

[0016] Figure 81 is a diagram of hardware configuration associated with determining ground object characteristics based on recording of vehicle events in accordance with at least some embodiments of the subject disclosure. DETAILED DESCRIPTION

[0017] The following disclosure provides many different embodiments or examples of various features of the subject matter provided. In order to simplify the present disclosure, the following records specific examples related to constituent elements, values, actions, materials, configurations or similar items. Of course, these are just examples and are not intended to be limiting. Other constituent elements, values, actions, materials, configurations or similar items can be assumed. For example, in the following description, forming a first feature above or on a second feature can include an embodiment in which the first feature is directly in contact with the second feature, and can also include an embodiment in which an additional feature can be formed between the first feature and the second feature in a manner that the first feature and the second feature may not be in direct contact. In addition, the present disclosure can repeat reference numbers and / or reference texts in various examples. The repetition is for the purpose of simplification and clarification, and the relationship between the various embodiments and / or structures described is not specified by the repetition itself.

[0018] Furthermore, spatially relative terms such as "below", "below", "below", "above", "above", and similar terms may be used in this specification to facilitate the description of the relationship of a certain element or feature relative to another element or feature as shown in the figures. In addition to the orientation depicted in the figures, spatially relative terms are intended to include various orientations of the device when in use or in action. The device may also be oriented in other situations (rotated 90 degrees or in other orientations), and the spatially relative descriptions used in this specification may also be interpreted accordingly.

[0019] Users such as software developers, insurance providers, market researchers, and law enforcement officers can use the On-Demand Data Retrieval (ODDR) system to input data requests to a user interface such as a graphical user interface (GUI). Software developers are, for example, software developers who develop applications, middleware, or OS (Operating Systems) executed on a vehicle. Exemplary applications include autonomous driving system applications, such as object recognition applications, road recognition applications, sensor fusion applications, positioning applications, route planning applications, controller applications, etc. The data request is analyzed and stored in the server, and then sent to the vehicle through the server. On the server side, the data request is stored in a storage unit, and a request queue is generated based on the stored requests. The user can confirm or request updates related to the status of the data request. For example, while the data request is still in the server before being sent to the vehicle, the status can be shown as "pending". When the server sends the data request to the vehicle, the status can be updated to "sent". Thus, the user can confirm and track the status of the data request made to the vehicle. In the description, a vehicle is indicated for clarity, but one skilled in the art will recognize that the description can be applied to a group of vehicles in addition to a single vehicle. The description can also be applied to non-vehicle computing devices equipped with at least one sensor and connected in some form.

[0020] The user interface for generating a data request includes a form associated with vehicle identification information, a requested data type, a start time, and an end time. In some embodiments, the requested data type includes a text field for syntax input, which enables more complex queries such as requesting data collected only during a "left turn at a red light". In some embodiments, the start time and the end time are absolute times, such as UNIX times that are elapsed time from a UNIX epoch time. In some embodiments, the start time and the end time are relative times relative to the time when the data request is received by the vehicle. In some embodiments, the start time and the end time are relative times relative to a triggering event. The triggering event is an event that occurs in the vehicle or in the environment surrounding the vehicle for which the user requests data, or the receipt of a data request implemented by the vehicle. For example, a triggering event generated from the environment surrounding the vehicle includes sudden acceleration, sudden braking, the acquisition of an image of the target of the data request, the detection of the target of the data request, or other suitable occurrences. User information for monitoring the status of the data request includes identification information of the data request and the status of the data request, such as reserved or sent.

[0021] In some embodiments, when a data request is received by a vehicle, the data request is processed in a manner that makes the data request agnostic about the source of the data request. In some embodiments, data request identification information (ID) is assigned to the received data request by the vehicle, such as by a request extraction unit in the vehicle. In some embodiments, the data request ID is assigned to the data request before the data request to the vehicle is sent. In some embodiments, the data request is generated by an application executed in the vehicle, and the application assigns the data request ID. In other words, the data is processed in a matching manner regardless of the program or system that sends the data request to the vehicle. In some embodiments, the data request is generated by a software component stored in the vehicle, and the data is processed matching the data request received from the external device. This helps to share the same data collection software component during triggered data collection, where the application generates a data collection request to a logger and an external data collection request based on ODDR.

[0022] In some embodiments, when a data request is received by a vehicle, the data request is processed in a manner that makes the data request agnostic to sensors and servers within the vehicle. In some embodiments, the data request is generated by an application executed in the vehicle. In some embodiments, an application programming interface (API) can be used to make the data request from the application agnostic to information from sensors or servers within the vehicle. This helps enable users to maximize data collection without having to program requests for specific sensor models. Next, the data request is transmitted to the data collection unit, and the requested data is collected in response to the occurrence of a triggering event. In the event that a triggering event such as a traffic accident has occurred, the data request is satisfied based on data stored in a storage device within the vehicle. The time frame for collecting data, i.e., the start time and the end time, is determined based on the data request. The collected data is transmitted back to the server.

[0023] Next, the collected data is stored in the server, and a notification about the completion of the data request is sent to the user. For example, the status of the data request is updated to "completed" on the user interface.

[0024] In some examples, a budget management system or payment system is implemented on the server side or the vehicle side in a manner that charges a user a fee for a data request. The fee can be paid at any time when the request is sent or when the data collection is completed. The fee can be adjusted based on the type and amount of data requested. In some embodiments, when the total amount of fees charged to the user reaches a maximum threshold of the user's budget, the data request from the user is rejected.

[0025] Through the ODDR system, users can access information collected by the vehicle in an on-demand manner. That is, data does not have to be collected continuously, and data can be collected in a manner that meets the specific needs of the user. In some embodiments, the ODDR system helps users such as software developers collect data and update the design, installation, and parameter adjustment of the user's software in an experimental manner based on the collected data, as a result, the user can continuously improve the software by, for example, communicating updates from a server to the vehicle via a network as an over-the-air (OTA) update. In some embodiments, the ODDR system helps machine learning developers who develop machine learning models for applications collect data and use data that was not available when the model was initially developed to train the model, as a result, the machine learning developer can update the model and continuously modify the weaknesses and problems of the model. In some examples, insurance providers can collect data associated with traffic accidents. In some examples, information associated with crimes or traffic accidents can be collected in law enforcement.

[0026] Vehicles subject to reactions triggered by extreme conditions, such as Antilock Brake System (ABS) engagement in response to strong force being applied to the brakes, seat belt locking and / or air bag deployment in response to extreme deceleration, traction control operation in response to wheel slip, other Operational Design Domain (ODD) limitations, etc., have information useful for warning other vehicles. However, the information currently used to determine the conditions and reactions themselves is insufficient to determine whether warnings should be issued to other vehicles and to which vehicles the warnings may apply.

[0027] For example, in the case of traction control operation, the information required for operation, such as the detection that all power is directed to the right wheel when the left wheel is stationary, may not be sufficient to determine whether the wheel slip is a special case or caused by ground characteristics such as weather events. Even in the case of slip caused by ground characteristics such as extreme weather, the information required for operation may not be sufficient to determine whether other vehicles are also experiencing wheel slip.

[0028] At least some embodiments of the subject disclosure utilize existing triggers that react in extreme situations to gather information sufficient to understand relevance to other vehicles, recording events from a sensor perspective, including sensors used to determine the present trigger and additional sensors.

[0029] Figure 11 is a schematic diagram of a request acquisition system 100 of a partial embodiment. The request acquisition system 100 includes a user interface (UI: User Interface) 110. The UI 110 is configured to receive a user request for data from a vehicle 140. The request acquisition system 100 also includes a server 120, and the server 120 is configured to: receive the user request from the UI 110, send the user request to the vehicle 140, receive data from the vehicle 140, and provide the data to the user via an accessible console 150. The server 120 includes a communication component 130 that communicates with the UI 110 and the vehicle 140. The request acquisition system 100 also includes an accessible console 150, which is configured to send the data collected from the vehicle 140 to the user.

[0030] UI110 is configured to receive input instructions from a user. In some embodiments, the user includes a software developer. In some embodiments, the user includes a machine learning model developer. In some embodiments, the user includes an insurance provider. In some embodiments, the user includes a law enforcement officer. In some embodiments, the user includes a market research company. UI110 provides the user with the option of selecting the type of vehicle requested and the type of data. In some embodiments, UI110 can generate a data request using a form associated with vehicle identification information, the type of data requested, a start time, and an end time. In some embodiments, the start time and the end time are absolute times, such as UNIX times as elapsed time from a UNIX time point. In some embodiments, the start time and the end time are relative times relative to the time when the data request is received by the vehicle. In some embodiments, the start time and the end time are relative times relative to a triggering event. In some embodiments, UI110 also provides the user with the option of selecting a triggering event and a duration of data collection relative to the triggering event. In some embodiments, UI110 includes information associated with the type of vehicle for which data is requested. In some embodiments, UI110 includes a vehicle ID that can uniquely identify the vehicle that is the target of the request. For example, the vehicle ID includes a Universally Unique Identifier (UUID) format. In some embodiments, UI110 includes data types that can identify the source of the data that the user wishes to collect. For example, the data types include the sensor ID of the sensor from which the sensor data is collected and the application ID of the application from which the application log is collected. In some embodiments, the format of the sensor ID and the application ID includes a Universally Unique Identifier (UUID) format. In some embodiments, UI110 includes a drop-down menu. In some embodiments, UI110 includes an editable field for accepting information associated with a data request. In some embodiments, UI110 includes a field for input related to a query language such as SQL (Structure Query Language). In some embodiments, UI110 provides information related to what types of data options are available to the user. In some embodiments, the types of data options available depend on the user. For example, in some embodiments, more data options can be selected in law enforcement than in insurance providers.

[0031] In some embodiments, UI110 includes a graphical user interface (GUI). In some embodiments, UI110 includes a mobile terminal that can be connected to server 120, such as a portable phone. In some embodiments, UI110 includes a network interface such as a RESTful API. In some embodiments, UI110 includes a computer that can be connected to server 120. In some embodiments, UI110 can be connected to server 120 wirelessly. In some embodiments, UI110 can be connected to server 120 via a wired connection. UI110 can also provide updates related to the status of data requests to the user. In some embodiments, UI110 provides status updates related to data requests in response to additional queries implemented by the user. In some embodiments, when update information is received from server 120, UI110 automatically provides status updates related to data requests without interaction with the user. In some embodiments, due to the status update, UI110 triggers a warning to the user. In some embodiments, the warning includes a voice warning or a visual warning.

[0032] In some embodiments, UI 110 includes a unit for accepting payment of fees from a user. In some embodiments, UI 110 includes a data entry field that enables a user to enter payment card information. In some embodiments, UI 110 includes a reader that detects payment card information, such as a magnetic stripe reader, a barcode reader, a chip reader, or other suitable reader.

[0033] The server 120 includes a communication component 130 configured to communicate with the UI 110 and the vehicle 140. The communication component 130 includes a receiver 131 configured to receive a data request from the UI 110. In some embodiments, the receiver 131 includes a wireless receiver. In some embodiments, the receiver 131 is configured to receive the data request via a wired connection. In some embodiments, the receiver 131 is also configured to initially process the received data request. In some embodiments, the received data request contains priority information. In some embodiments, the receiver 131 is configured to assign a priority to the data request based on the identity of the user who sent the data request or the fee paid by the user who sent the data request. In some embodiments, the receiver 131 is configured to assign a request identification information (ID) number to each received data request. In some embodiments, the server 120 is configured to limit access to specific sensors in the vehicle 140 based on the identity of the user. For example, in some embodiments, a third-party user cannot access sensors associated with the safety function of the vehicle 140.

[0034] The communication component 130 also includes a memory unit 132 configured to store data requests received by the receiver 131. In some embodiments, the memory unit 132 includes a random access memory, a solid-state memory, or other types of memory. In some embodiments, the memory unit 132 is configured to store the data request together with the status of the data request. In some embodiments, the status of the data request includes reserved (before sending the data request to the vehicle 140), sent (after sending the data request to the vehicle 140), and completed (after receiving the request data from the vehicle 140). In some embodiments, the memory unit 132 is accessible by a user. In some embodiments, an update to the information in the memory unit 132 triggers a notification to the user associated with the information updated in the memory unit 132. In some embodiments, the memory unit 132 stores the data request together with timestamp data indicating the time when the data request was received. In some embodiments, the memory unit 132 stores the data request in association with a priority. In some embodiments, the priority is determined based on the identity of the user. For example, in some embodiments, law enforcement has a higher priority than an insurance provider, which has a higher priority than a typical user such as a software developer. In some embodiments, the priority is determined based on a fee paid by the user. For example, in some embodiments, a user can pay a fee to increase the priority of the user's request so that the requested data can be obtained more quickly. In some embodiments, the priority and the amount of the fee are determined based on dynamic price settings. In some embodiments, the amount of the fee is proportional to the priority. In some embodiments, the priority and the amount of the fee are determined based on a respectively set price limit, an upper limit, and an analysis of resource availability. In some embodiments, the priority of a data request increases as the amount of time between the initial storage of the data request and the sending of the data request to the vehicle increases.

[0035] The communication component 130 also includes a transmitter 133. The transmitter 133 is configured to send the status of the data request to the UI 110. In some embodiments, the status of the data request is sent to the UI 110 wirelessly. In some embodiments, for example, when the vehicle is connected at a dealer, the status of the data request is sent to the UI 110 via a wired connection. In some embodiments, the transmitter 133 is configured to automatically provide updates to the data request in response to updates in the memory unit 132. In some embodiments, the transmitter 133 is configured to provide updates to the data request in response to an update request received from a user. In some embodiments, the transmitter 133 is configured to automatically send the request ID when the data request is initially saved in the memory unit 132. In some embodiments, the status of the data request includes a priority of the data request. In some embodiments, the status of the data request includes an estimated time until the data request is sent to the vehicle 140.

[0036] The communication component 130 also includes a query queue 134 configured to store data requests in a priority order for transmission to the vehicle 140. In some embodiments, the query queue 134 is embedded in the memory unit 132. In some embodiments, the query queue 134 is separate from the memory unit 132. In some embodiments, the query queue 134 is configured to obtain data requests from the memory unit 132 based on priority and timestamp information. In some embodiments, the query queue 134 is configured to sort the data requests based on priority, and in response to data requests having the same priority, sort the data requests according to the time since they were initially stored in the memory unit 132.

[0037] The communication component 130 also includes a transmitter 135 configured to send a data request from the query queue 134 to the vehicle 140. The transmitter 135 is configured to send the data request to the vehicle 140 based on the order of the data requests in the query queue 134. In some embodiments, the data request is sent to the vehicle 140 wirelessly. In some embodiments, the data request is sent to the vehicle 140 through a wired connection. The data request sent to the vehicle 140 includes: trigger event information; data duration information associated with a period before and after the trigger event when data should be collected; and sensor information indicating that a certain type of sensor of the vehicle 140 should collect data. In some embodiments, the data request sent to the vehicle 140 includes priority information. In some embodiments, the transmitter 135 is configured to: when the vehicle 140 sends a request to the server 120 to send the data request to the vehicle 140, the transmitter 135 sends the data request to the vehicle 140. In some embodiments, the transmitter 135 is configured to send a data request to the vehicle 140 at any time when the communication component 130 has sufficient connectivity with the vehicle 140 to send the data request, as long as the communication component 130 has not received information indicating that the vehicle 140 cannot accept a new data request. In some embodiments, the transmitter 135 is configured to send the data request to the vehicle 140 periodically as long as the vehicle 140 can receive new data requests and the transmitter 135 has sufficient connectivity with the vehicle 140. In some embodiments, the transmitter 135 is configured to send the data request to the vehicle 140 in batches, such as in groups of five data requests, twenty data requests, or some other number of data requests. In some embodiments, the transmitter 135 is configured to request a confirmation of receipt of the data request from the vehicle 140. In some embodiments, the transmitter 135 is configured to resend the data request in response to not receiving a confirmation of receipt from the vehicle within a specified period of time. In some embodiments, the status of the data request stored in the memory unit 132 is updated to indicate transmission to the vehicle 140 in response to the communication component 130 receiving an acknowledgment from the vehicle 140 regarding receipt of the data request.

[0038] The communication component 130 also includes a receiver 136 configured to receive a notification of the occurrence of a triggering event from the vehicle 140. In some embodiments, the occurrence of the triggering event is the receipt of a data request. In some embodiments, the receiver 136 is configured to receive the notification of the triggering event wirelessly. In some embodiments, the receiver 136 is configured to receive the notification of the triggering event via a wired connection. In some embodiments, the receiver 136 is configured to send a signal to the memory unit 132 to update the status of the data request associated with the notified triggering event.

[0039] The communication part 130 also includes a receiver 137 configured to receive data from the vehicle 140 in response to the data request sent by the transmitter 135. In some embodiments, the data is divided into data packets by the vehicle 140, which are units of transmission from the vehicle 140 to the server 120, and the receiver 137 receives the data packets from the vehicle 140. In some embodiments, the receiver 137 is configured to receive the data wirelessly. In some embodiments, the receiver 137 is configured to receive the data via a wired connection. In some embodiments, the receiver 137 is configured to send a signal to the memory unit 132 that updates the status of the data request associated with the reception of the request data. In some embodiments, the data in response to a single data request is received from the vehicle 140 in a single packet. In some embodiments, the data in response to a single data request is received from the vehicle 140 in multiple packets. The receiver 137 transmits the received data to the preprocessor 122.

[0040] The server 120 also includes a preprocessor 122 configured to receive data from the receiver 137 and preprocess the data to generate collected data. In some embodiments, preprocessing includes modifying data from multiple packets in response to a data request in a manner that compiles the data. In some embodiments, preprocessing includes deserializing the data in a manner that compiles structured data from a received byte array. In some embodiments, preprocessing includes decompressing the data if the data is compressed by the vehicle 140 before transmission. In some embodiments, preprocessing includes error correction implemented by error correction codes (ECC: Error Correction Code) such as Reed-Solomon (RS) codes, Bose-Chaudhuri-Hocquenghem (BCH) codes, low-density parity-check (LDPC) codes, and the like. In some embodiments, preprocessing includes smoothing the data by removing outliers to reduce the risk of reporting erroneous data to the user. In some embodiments, preprocessing includes associating data request ID information, priority information, or other suitable information with the received data from the receiver 137. In some embodiments, preprocessing includes aggregating query results from more than one vehicle. In some embodiments, preprocessing includes offloading one or more processes from the server. In some embodiments, the data is preprocessed in such a way that the information is provided to the client in a format that is easily understood and does not rely on specialized knowledge or equipment to discern the information.

[0041] The server 120 also includes a data storage 126 configured to store the collected data generated by the preprocessor 122. In some embodiments, the data storage 126 is embedded in the memory unit 132. In some embodiments, the data storage 126 is separate from the memory unit 132. In some embodiments, the data storage 126 includes a solid state drive (SSD), a random access memory, or other suitable memory. In some embodiments, the data storage 126 can be accessed by a user using the UI 110 or the accessible console 150, for example. In some embodiments, the data storage 126 is configured to notify the user in response to the data associated with the data request being available. In some embodiments, the notification includes a warning to the user. In some embodiments, the warning includes a voice warning or a visual warning. In some embodiments, the data storage 126 is configured to cause the UI 110 or the accessible console 150 to automatically display a notification of the availability of the collected data. In some embodiments, the data storage 126 can be accessed by the user using the accessible console 150 without the user sending a data request. In some embodiments, the data in the data storage 126 can be retrieved by a user via an accessible console 150. In some embodiments, the collected data is visualized in the console 150.

[0042] The request acquisition system 100 also includes a vehicle 140. The vehicle 140 includes sensors for detecting both the internal conditions of the vehicle 140 and the external environment around the vehicle 140. In some embodiments, the sensors include cameras, light ranging (LiDAR) sensors, wireless ranging (RADAR) sensors, sound-based navigation and ranging (SONAR) sensors, accelerometers, steering wheel positions, speedometers, or other suitable sensors. The vehicle 140 can receive data requests through either wireless or wired connections.

[0043] In some embodiments, the vehicle 140 is configured to assign a data request ID to the received data request in response to receiving the data request, and the data request is processed in a manner that is agnostic to the system or program that generated the data request. In another embodiment, the communication component 130 assigns the data request ID on behalf of the vehicle 140, and the data request ID is included in the data request sent from the communication component 130 to the vehicle 140. Making the data request agnostic to the system or program that generated the data request helps expand the ability of the vehicle 140 to receive and process a wide range of data requests from various users and systems. The vehicle 140 includes a processor that processes the data request and determines the type of information that the sensors available in the vehicle 140 can satisfy the data request. In at least some embodiments, the vehicle 140 includes a mobile computing network, which is a network of processors, controllers, or a combination thereof, such as a controller area network (CAN). In at least some embodiments, each processor is an electronic control unit (ECU: Electronic Control Unit). In at least some embodiments, there is a microcontroller unit (MCU: Microcontroller Unit). The vehicle 140 also includes a memory that stores data from the sensor. In some embodiments, the processor accesses the memory to determine whether any stored data can satisfy the data request. The vehicle 140 can also send the data deemed to satisfy the data request to the server 120 via either a wireless or wired connection. In some embodiments, the processor is configured to attempt to satisfy the received data requests in a priority order based on the reception priority of the data requests. In some embodiments, the vehicle 140 is configured to preferentially send the data to the server based on the reception priority of the data requests.

[0044] In some embodiments, the memory and processor of the vehicle 140 are configured to store and execute software applications in an electronic control unit (ECU) within the vehicle 140. In some embodiments, the data request is generated by the software application stored in the ECU. In some embodiments, the data request is generated in response to a trigger event, such as sudden acceleration, sudden braking, the acquisition of sensor data of a specific object or a specific scene predetermined in the software application, a "crash" of the software application, an abnormality detected in the software application, or other suitable detection events. In some embodiments, the vehicle 140 is configured to generate a notification to the maintainer of the software application, such as a user, in response to the detection of a trigger event associated with the software application. In some embodiments, the notification is sent directly to the user, such as through the UI 110, through any one of wireless or wired connections. In some embodiments, the notification is sent to the user through the server 120, through any one of wireless or wired connections. In some embodiments, the notification includes a voice notification or a visual notification. In some embodiments, the notification is configured to cause the UI 110 to automatically display the notification without interaction with the user.

[0045] The request acquisition system 100 also includes an accessible console 150. The accessible console 150 enables a user to access the collected data stored in the data storage 126. In some embodiments, the accessible console 150 is embedded in the UI 110. In some embodiments, the accessible console 150 is separate from the UI 110. In some embodiments, the accessible console 150 includes another server separate from the server 120. In some embodiments, when the data storage 126 receives the collected data, the accessible console 150 automatically receives the collected data associated with the data request from the user. In some embodiments, the accessible console 150 enables the user to search the data storage 126 to determine whether any of the collected data stored in the data storage 126 is useful to the user without the user sending a data request.

[0046] Use of the request capture system 100 enables users to obtain information from one or more vehicles 140 in a format that is easily understood without relying on specialized equipment to request or read received data. Data requests can be prioritized in the request capture system 100, thereby facilitating users to pay to obtain data more quickly and ensuring that law enforcement or other users can obtain data. This flexibility helps improve the usefulness of the request capture system 100 to a wide range of users.

[0047] Figure 2FIG. 2 is a diagram of a graphical user interface (GUI) 200 and 250 for a request acquisition system according to some embodiments. In some embodiments, GUI 200 can be used as a graphical user interface for request acquisition system 100 ( Figure 1 ) in the UI 110. In some embodiments, the GUI 200 may be used to generate the receiver 131 ( Figure 1 ) to receive data requests. GUI 200 includes a plurality of information types 210 that identify the types of information that GUI 200 can accept from the user. GUI 200 also includes a plurality of fields 220 configured to accept information associated with the corresponding information types 210 of GUI 200. GUI 200 also includes a send button 230 that is configured to send the data request to a server, such as server 120 (based on the information in field 220). Figure 1 ). Those skilled in the art will recognize that the names and quantities of the plurality of information types 210 are merely examples, and that different quantities and types of information are also within the scope of the present disclosure.

[0048] In some embodiments, field 220 includes fields for the user to enter a vehicle ID, a data type, a start time, and an end time. In some embodiments, field 220 also includes a field for the user to enter a priority for the data request. In some embodiments, GUI 200 also includes information associated with a method by which the user can increase the priority of the data request, such as information indicating the fees associated with each available priority. In some embodiments, GUI 200 includes field 220 that allows the user to enter login information to establish the identity of the user. In some embodiments, GUI 200 is configured to display the user's priority after receiving the login information. In some embodiments, GUI 200 also includes field 220 for accepting payment information associated with the fees used to establish the priority of the data request. In some embodiments, field 220 includes a field for input related to a query language such as SQL. In some embodiments, GUI 200 is configured to use real samples or made samples to try and plan queries and display sample data. In some embodiments, the GUI 200 is configured to utilize a “REPL (Read-Eval-Print Loop)” such as SciPy / NumPy / Matplotlib or R, or a console such as Jupyter notebook.

[0049] The GUI 250 is configured to be displayed to the user after the user selects the send button 230 on the GUI 200. In some embodiments, the GUI 250 can be used as an ODDR system 100 ( Figure 1) in the UI 110. The GUI 250 includes information indicating that the data request has been received. The GUI 250 includes a query ID tag 260 and a query ID field 270. After the server receives and stores the data request, the server, such as the server 120 ( Figure 1 ) receives information input into the query ID field 270. In some embodiments, the GUI 250 includes information about the vehicle ID. In some embodiments, the GUI 250 includes information associated with the priority of the data request. In some embodiments, the GUI 250 includes information related to the status of the data request, such as reserved, sent, completed, etc. In some embodiments, the GUI 250 includes information related to the status of the data request until the data request is sent to the vehicle, such as the vehicle 140 ( Figure 1 ). In at least some embodiments, GUI250 includes information associated with an estimated time until the requested data is received. In at least some embodiments, GUI250 includes information associated with an estimated energy consumption for receiving the requested data. In some embodiments, GUI250 is automatically displayed in response to receipt of query ID information from the server. In some embodiments, GUI250 is displayed in response to a user sending a request for an update to the uploaded data.

[0050] Figure 3 FIG. 1 is a diagram of a data structure 300 of a request acquisition command 310 according to some embodiments. In some embodiments, the request acquisition command 310 is sent from the server 120 to the vehicle 140 ( Figure 1 ). The request acquisition command 310 includes a request for a vehicle, such as vehicle 140 ( Figure 1 ) data request contains information associated with the type of data requested.

[0051] The request to obtain command 310 includes a transmission priority parameter 311 indicating the priority of the data request. If so, the request to obtain command 310 also includes a log level parameter 312 indicating the type of data that should be obtained from other applications in the vehicle. For example, in some embodiments, the request to obtain command 310 obtains data from an object recognition application. The log level parameter 312 determines the type of data obtained from other applications, such as an error level or a major level. In some embodiments, the log level parameter 312 is omitted from the request to obtain command 310, or the log level parameter 312 is set to a null state. The request to obtain command 310 also includes a collection time range parameter 313 indicating the period before and / or after the triggering event for collecting data. In some embodiments, the time range is consistent with the time range input by the user to the GUI 200 ( Figure 2) corresponds to the start time and end time of the data request. The request acquisition command 310 also includes a Uniform Resource Locator (URL) endpoint parameter 314 indicating the destination of the data collected in response to the data request. If so, the request acquisition command 310 also includes a frequency parameter 315 indicating the frequency at which the data should be sampled from the collection time range parameter 313. For example, when the event time is t=100 seconds, and the time range has a start time=-1 second and an end time=2 seconds, and the frequency is 10Hz (100 millisecond cycle), data at t=99.0 seconds, 99.1 seconds, 99.2 seconds,..., 101.9 seconds, and 102.0 seconds are collected by the request acquisition command. The request acquisition command 310 also includes a log ID parameter 316 indicating the type of sensor and / or application that can be used to collect the data requested by the data request. In some embodiments, a unique ID (such as a universal unique identifier (UUID)) is pre-assigned to all sensors and applications, and the unique ID of the user who wants to collect data is specified in the log ID parameter 316. The request acquisition command 310 also includes a requester ID parameter 317 indicating the identity of the user making the data request. The request acquisition command 310 also includes an event ID parameter 318 indicating the triggering event associated with the data request. The request acquisition command 310 also includes an event ID parameter 318 indicating the vehicle that should be assigned to satisfy the data request, such as vehicle 140 ( Figure 1 ) of the resource budget ID parameter 319. Those skilled in the art will appreciate that additional parameters may be included in the request acquisition command 310. For example, in some embodiments, the request acquisition command 310 includes a vehicle location parameter indicating a geographic area where the trigger event can occur. Those skilled in the art will also appreciate that the request acquisition command 310 does not always include a vehicle location parameter. Figure 3 For example, in some embodiments, the budget ID parameter 319 is omitted.

[0052] Figure 4 4 is a block diagram of a request acquisition system 400 according to a certain embodiment. In certain embodiments, the request acquisition system 400 is a block diagram of a request acquisition system 100 ( Figure 1 ) is a part of. In some embodiments, the request acquisition system 400 can be combined with the request acquisition system 100 ( Figure 1 In some embodiments, the request acquisition system 400 is used together with the request acquisition system 100 ( Figure 1 ) separated.

[0053] The request acquisition system 400 includes a detection vehicle system 410 configured to acquire information related to the vehicle or the surroundings of the vehicle. The detection vehicle system 410 acquires information related to the vehicle and the surroundings and sends the information to a server. The request acquisition system 400 also includes a server 440, which is configured to receive the information, encode the information, and transmit the information to a user terminal 460.

[0054] The detection vehicle system 410 includes an electronic control unit (ECU) 420 configured to receive data from a sensor 414, a global positioning system (GPS) 416, and a map 418. The ECU 420 includes a condition detector 422, a data designator 432, a log collector 434, and a log transmitter 436. The condition detector 422 includes a vehicle control monitor 424, an object detector 426, and a scene detector 428.

[0055] In some embodiments, the ECU 420 also includes a localization unit that is configured to receive data from the GPS 416 and the map 418 and determine the position of the vehicle relative to the position of detected and / or known objects and / or roads and the posture and state of the vehicle. The posture is the orientation of the vehicle relative to a reference point such as a lane. In some embodiments, the position of the vehicle also refers to the position vector of the vehicle. The posture and state of the vehicle refer to the speed and direction of travel of the vehicle. In some embodiments, the posture and state of the vehicle also refer to the velocity vector, acceleration vector, and jerk vector of the vehicle. In some embodiments, the position vector, velocity vector, acceleration vector, and jerk vector include an angular vector. In some embodiments, the state of the vehicle also refers to whether the engine or motor of the vehicle is working.

[0056] The sensor 414 is configured to take in information such as images of the environment around the vehicle. In some embodiments, the sensor 414 includes a visible light camera, an IR (Infrared) camera. In some embodiments, the sensor 414 is replaced by a light detection and ranging (LiDAR) sensor, a wireless detection and ranging (RADAR) sensor, a sound-based navigation and ranging (SONAR) sensor or other suitable sensors, or further accompanied by them. In some embodiments, the sensor 414 includes additional cameras configured in other places in the vehicle. For example, in some embodiments, the additional camera is configured on the side of the vehicle to detect a larger part of the environment located on the left and right of the observation vehicle. The occupants of the vehicle can see outside from the side windows of the vehicle, so using additional cameras to detect a larger part of the environment around the vehicle helps to increase the accuracy of detecting objects or scenes around the vehicle. For example, in some embodiments, the additional camera is configured on the rear side of the vehicle to detect a larger part of the environment located behind the vehicle. This information helps to retrieve information related to the object. In some embodiments, the data from the sensor 414 includes a timestamp or other metadata to help synchronize the data from the sensor 414 with data from other components.

[0057] GPS 416 is configured to determine the location of the vehicle. Knowing the location of the observed vehicle helps to associate an object or scene with the determined location on map 418.

[0058] Map 418 contains information associated with lanes and known objects along the lanes. In some embodiments, map 418 can be used with GPS 416 to determine the location and direction of travel of the vehicle. In some embodiments, map 418 is received from an external device such as server 440. In some embodiments, map 418 is regularly updated based on information from sensor 414 and / or GPS 416. In some embodiments, map 418 is regularly updated based on information received from an external device. In some embodiments, map 418 is generated from sensor data through a SLAM (Simultaneous Localization And Mapping) algorithm. Including map 418 helps determine whether an object is a known object. Including map 418 with known objects helps increase the accuracy of new object detection.

[0059] The condition detector 422 is configured to generate information associated with the performance of the vehicle and the performance of the systems within the vehicle. The condition detector 422 can collect information from components within the vehicle, such as sensors 414, braking systems, acceleration systems, and other suitable components. The condition detector 422 can use this information to determine the performance of the vehicle. In some embodiments, the condition detector 422 is also configured to monitor the performance of software and networking actions within the vehicle. For example, in some embodiments, the condition detector 422 is configured to receive information associated with a "crash" of software or applications within the vehicle. In some embodiments, the condition detector 422 is configured to collect information related to the storage capacity of a memory device within the vehicle. In some embodiments, the condition detector 422 is configured to receive information associated with the processing power of a processor within the vehicle.

[0060] The vehicle control monitor 424 is configured to receive sensor data and control logs associated with the current action of the vehicle. In some embodiments, the sensor data includes information associated with vehicle speed, acceleration, jerk, braking, steering, pitch, roll, yaw, flashing of hazard lights, sirens, or other suitable information. The vehicle control monitor 424 is configured to determine whether any of the received sensor data indicates that a benchmark for satisfying the request has been met, such as whether a triggering event has been detected.

[0061] The object detector 426 is configured to receive sensor data from the sensor 414 to determine whether any abnormal object is located in the lane. In some embodiments, the object detector 426 is also configured to determine whether any object exists along the lane or adjacent to the lane. In some embodiments, the sensor data from the sensor 414 includes an image, and the object detector 426 is configured to perform image recognition on the received image, for example, using a trained neural network, to identify abnormal objects. In some embodiments, the object detector 426 is configured to compare any identified object with information from the GPS 416 and the map 418, thereby helping to determine the type of the identified object. In some embodiments, the object detector 426 is configured to identify objects such as tires, car parts, animals, potholes, traffic control panels, emergency vehicles, vehicles with activated hazard lights, or other suitable objects as objects.

[0062] The scene detector 428 is configured to receive sensor data from the sensor 414 to determine whether any scene that satisfies the conditions for satisfying the request is located in the environment surrounding the vehicle. In some embodiments, the scene detector 428 is configured to determine that a vehicle accident has occurred in response to the detection of two or more vehicles in contact with each other or a vehicle surrounded by multiple dropped objects. In some embodiments, the scene detector 428 is configured to determine that construction is in progress based on the detection of multiple construction vehicles approaching. In some embodiments, the scene detector 428 is configured to determine that the vehicle is parked on the shoulder of the road based on a determination that the vehicle is located adjacent to a lane and is not moving or is moving significantly slower than other vehicles. In some embodiments, the scene detector 428 is configured to use image recognition, such as by a trained neural network, to determine the content of the scene surrounding the vehicle.

[0063] In some embodiments, object detector 426 and scene detector 428 are each activated during the entire operation of the vehicle, such as when the engine or motor of the vehicle is running. In some embodiments, at least one of object detector 426 or scene detector 428 is set to be activated in response to vehicle control monitor 424 determining a specific behavior, such as a trigger event being detected.

[0064] The data designation unit 432 is configured to receive a determination that the implementation of the request has been completed or a determination that a triggering event has been detected. The data designation unit 432 is configured to analyze the received information and determine what sensor data from the sensor 414 should be collected based on the received data. For example, in some embodiments where an abnormal steering behavior implemented by the driver is detected, the data designation unit 432 is configured to determine that image data from the front camera of the sensor 414 should be taken in. Moreover, the data designation unit 432 is configured to determine the period during which data from the determined sensor should be collected based on the time of the detection condition. In some embodiments, the data designation unit 432 is configured to determine the sensor 414 for collecting data based on instructions in the request received from the user.

[0065] In some embodiments, the data designation unit 432 is configured to determine an area of ​​received sensor data associated with a detected condition. In some embodiments, the area of ​​received sensor data is identified based on, for example, object recognition of the sensor data by the object detector 426 or the scene detector 428. In some embodiments, the data designation unit 432 is configured to intercept a received image from the sensor data or remove data of a different nature from the sensor data if the sensor data is not an image to reduce the amount of information in the log of the abnormal condition. In some embodiments, the data designation unit 432 is configured to remove personal information, such as license plates, faces of people, etc., from the sensor data.

[0066] The log collector 434 is configured to receive data from the data designator 432. In some embodiments, the log collector 434 is configured to receive data directly from the sensor 414, GPS 416, or the condition detector 422 based on information provided by the data designator 432. The log collector 434 is also configured to determine what information is useful for identifying the type and location of the object, such as location information from the GPS 416 or the map 418, image information from the sensor 414, truncated or reduced information from the data designator 432, timestamp information associated with the time when the object or scene was detected, or other suitable information.

[0067] The log collector 434 generates log data based on received and related data, such as captured images and location data. To support synchronization of the collected data and for queue priority within the server 440, the log collector 434 also associates timestamp information with the log data. In some embodiments, the log collector 434 generates log data in a manner that further includes world coordinates associated with the captured images. In some embodiments, the log collector 434 generates log data in a manner that further includes map locations associated with the captured images. In some embodiments, the log collector 434 includes additional information to support increased accuracy in determining objects or scenes.

[0068] The above description involves generating log data based on images from sensor 414, but those skilled in the art will appreciate that log collector 434 is not limited to generating log data based on images. In some embodiments, log collector 434 is configured to generate log data based on information from other sensors mounted on the vehicle, such as RADAR, LiDAR, or other suitable sensors. In some embodiments where the occupant wears smart glasses, log collector 434 is also configured to generate log data based on information received from the smart glasses.

[0069] The log sender 436 is configured to receive log data from the log collector 434 and send the log data to the server 440. In some embodiments, the log sender 436 is configured to send log data wirelessly. In some embodiments, the log sender 436 is configured to send log data via a wired connection. In some embodiments, the log sender 436 is configured to send log data directly to the user terminal 460. In some embodiments, the log sender 436 is configured to send log data to a mobile device that can be accessed by a user, and then the mobile device is configured to send the log data to the server 440. In some embodiments, the log sender 436 is configured to use Bluetooth (Bluetooth) (registered trademark) or other suitable wireless technologies to send log data to the mobile device. In some embodiments, the ECU 420 is configured to determine whether the data transmission speed from the mobile device to the server 440 is higher than the data transmission speed from the log sender 436 to the server 440. The log sender 436 is configured to send the log data to be sent to the server 440 to the mobile device in response to a determination that the data transmission speed from the mobile device to the server 440 is higher. The log sender 436 is configured to send the log data directly from the detection vehicle system 410 to the server 440 without transmitting the log data to the mobile device in response to a determination that the data transmission speed from the mobile device to the server 440 is not high.

[0070] In some embodiments, the detection vehicle system 410 also includes a memory configured to store sensor data from sensors mounted on the vehicle. In some embodiments, the memory is also configured to store information associated with previously detected objects or scenes. In some embodiments, the data designation unit 432 is configured to provide a result based on the consistent object or scene in response to detecting an object or scene consistent with a previous object or scene. In some embodiments, the detection vehicle system 410 is also configured to determine whether the detection vehicle has received information associated with an object or scene consistent with the object or scene determined from the condition detector 422 from the server 440. In some embodiments, the detection vehicle system 410 is configured to hinder the sending of log data to the server 440 in response to detecting that the detection vehicle has received information associated with the determined object or scene. Avoiding the sending of redundant information to the server 440 helps reduce the data sent to the server 440 and helps minimize the power consumption of the detection vehicle system 410. In some embodiments, the storage of previous requests is called a cache. Those skilled in the art will understand caching as the use of hardware or software to store data so that future requests for the data can be satisfied more quickly.

[0071] The server 440 includes a log data receiver 442 configured to receive log data from the log sender 436. In some embodiments, the log data receiver 442 is configured to receive log data from a mobile device. The server 440 also includes a log encoder 444 configured to encode the log data. The server 440 also includes a log transmitter 446 configured to send the encoded log data to the user terminal 460. The server 440 also includes a request / rule receiver 448 configured to receive a request or rule from the user terminal 460.

[0072] The log data receiver 442 is configured to receive log data from the log transmitter 436. In some embodiments, the log data receiver 442 is configured to receive log data from a mobile device. In some embodiments, the log data receiver 442 is configured to receive log data wirelessly. In some embodiments, the log data receiver 442 is configured to receive log data via a wired connection. In some embodiments, the log data receiver 442 is configured to attach a timestamp to the log data regarding the time when the log data was received.

[0073] The log encoder 444 is configured to encode the received log data according to a specified encoding protocol. Encoding the log data according to the specified encoding protocol helps to ensure that the user terminal 460 can reliably decode the log data for use by the user terminal 460. In some embodiments, the log encoder 444 is configured to perform compression, image encoding, thumbnail image generation, or other suitable encoding protocols for the log data. In some embodiments, the log encoder 444 is configured to encrypt the log data. In some embodiments, the log encoder 444 is also configured to perform super-resolution so that the user can further identify the data. It should be understood by those skilled in the art that super-resolution is the process of receiving a high-resolution image from a low-resolution image. Improving the resolution of log data helps to reduce false positives or false negatives.

[0074] In some embodiments, the server 440 also includes a database for storing received log data. In some embodiments, the log data is stored in the database before and / or after encoding by the log encoder 444. In some embodiments, the log data is stored in the database in a priority queue. In some embodiments, the priority of the priority queue is determined based on an object or scene, such as the time when the triggering event is detected, the time when the log data is received by the log data receiver 442, the type of object or scene, the identity of the driver of the detecting vehicle, or other suitable priority criteria.

[0075] The log transmitter 446 is configured to receive the encoded log data from the log encoder 444. The log transmitter 446 is configured to send the encoded log data to the user terminal 460. In some embodiments, the log transmitter 446 is configured to send the encoded log data to a mobile device that can be accessed by a user. In some embodiments, the log transmitter 446 is configured to send the encoded log data wirelessly. In some embodiments, the log transmitter 446 is configured to send the encoded log data via a wired connection. In some embodiments, the log transmitter 446 is configured to send the encoding protocol information together with the encoded log data. The transmission of the encoding protocol information about the encoded log data helps the mobile device or the user terminal 460 to accurately decode the encoded log data for use by the user terminal 460.

[0076] The request / rule receiver 448 is configured to receive new or updated rules or requests for data from a user. In some embodiments, the request / rule receiver 448 is configured to receive new or updated rules or requests wirelessly. In some embodiments, the request / rule receiver 448 is configured to receive new or updated rules or requests via a wired connection. In some embodiments, the request / rule receiver 448 is based on the UI 110 ( Figure 1 ) device.

[0077] In some embodiments, the server 440 is configured to receive formed locations from a plurality of vehicles. In some embodiments, the server 440 is configured to receive navigation plans from a plurality of vehicles. In some embodiments, the log transmitter 446 is configured to limit the transmission of the encoded log data only to vehicles within a specified distance of a detected triggering event.

[0078] In some embodiments, the server 440 is configured to send only log data associated with a newly detected trigger event. That is, if the trigger event has already been reported by the server 440, the trigger event is not reported again. Limiting repeated reporting of trigger events helps reduce redundant data received by the user terminal to the server 440.

[0079] The user terminal 460 is a user terminal accessible by a user and associated with a satisfied request. In some embodiments, the user terminal 460 includes a GUI. In some embodiments, the user terminal 460 is configured to automatically generate an alert in response to receiving data from the server 440. In some embodiments, the alert includes a voice alert or a visual alert.

[0080] Those skilled in the art will appreciate that modifications to the request acquisition system 400 are within the scope of the present disclosure. For example, in some embodiments, the detection vehicle system 410 can send log data directly to the user terminal 460 over a wireless network or the like. In some embodiments, the mobile device of the passenger in the detection vehicle can send log data directly to the user terminal 460 over a wireless network or the like.

[0081] By automatically identifying and transmitting information associated with satisfying rules or requests detected in or around the vehicle, a user can improve the performance of applications or software executed using the vehicle's processing system, such as ECU 420. In some embodiments, a user can object to information associated with an event such as an accident.

[0082] Figure 5 The present invention provides a method for determining the characteristics of a ground object based on a record of a vehicle event. In at least some embodiments, the method is performed by a vehicle, such as Figure 1 The vehicle 140 or Figure 4 The detection vehicle system 410 is shown. In at least some embodiments, the method is performed by Figure 4 The ECU 420 shown detects the controller of the vehicle system or Figure 8 The controller 802 is shown to perform.

[0083] In S550, the detection component of the controller detects a vehicle reaction. In at least some embodiments, the detection component detects a vehicle reaction of the vehicle, and the vehicle reaction is triggered by a reading threshold of at least one trigger sensor of the vehicle. In at least some embodiments, the reading threshold is based on the operational design domain (ODD). In at least some embodiments, when the vehicle approaches the ODD limit of at least one trigger sensor, a vehicle action is triggered, such as a safety mechanism, ABS, airbag, etc. In at least some embodiments, an existing trigger can be used as a basis for recording an event. Moreover, in at least some embodiments, a specific query from a programmer can be used as a trigger. In at least some embodiments, the programmer can specify sensors to be recorded and sensors not to be recorded, respectively, in response to a recording trigger defined by the programmer. In at least some embodiments, the detection component adjusts the reading threshold based on the amount of available storage. In at least some embodiments, the detection component performs the following description Figure 6 Action shown.

[0084] In S552, the recording component of the controller records the output of the sensor. In at least some embodiments, the recording component records the output of at least one trigger sensor and at least one associated sensor of the vehicle in response to the detection of the vehicle reaction. In at least some embodiments, at least one associated sensor has a GPS sensor. In at least some embodiments, the associated sensors include one or more of a sensor for detecting a gear ratio in an event of a vehicle reaction triggered by wheel slip, a temperature sensor, a rain detection sensor, a snow detection sensor, a speed sensor, a tire wear estimation sensor, a GPS sensor, a driver's perspective camera, and the like. In at least some embodiments, the recording component performs the following recording Figure 7 action flow.

[0085] In S554, the determination component of the controller determines the target characteristic of the ground. In at least some embodiments, the determination component determines the target characteristic of the ground based on the recorded output associated with each of the at least one triggering sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle. In at least some embodiments, the at least one characteristic of the vehicle has at least one of a static characteristic based on the make and model of the vehicle or a dynamic characteristic of the vehicle during the recorded time. In at least some embodiments, the recorded information of the event is analyzed in a manner that normalizes the state associated with the vehicle characteristics including static characteristics, dynamic characteristics, or both, wherein the static characteristics are, for example, vehicle weight, ground clearance, gear ratio, drive system, etc., and the dynamic characteristics are, for example, load weight, tire pressure, current gear, current speed, etc. In at least some embodiments, for example, when the ground clearance sensor detects that the ground has reached 80% of the clearance, 80% is compared with the known maximum clearance height associated with the model of the vehicle, thereby converting 80% into an actual height. As a result, the target characteristic of the ground is effectively determined independently of the vehicle that recorded the event. In at least some embodiments, for example, in the event of a vehicle reaction triggered by wheel slip, the information may specify that the ground is frozen road, taking into account temperature and detection of rain or snow. In at least some embodiments, the information specified about the ground itself may be useful even if there is only one instance. However, in at least some embodiments, the information specified about the ground itself may be improved by aggregating events recorded from many vehicles in the same geographic location.

[0086] In S556, the sending component of the controller sends the target characteristic. In at least some embodiments, the sending component sends the target characteristic to a server via a network. In at least some embodiments, the sending component sends the target characteristic directly to another vehicle.

[0087] Figure 6 The present invention is a flow chart of detecting a vehicle reaction in at least some embodiments of the subject disclosure. The flow chart provides a method for detecting a vehicle reaction. In at least some embodiments, the detection component adjusts the read threshold based on the amount of available storage. In at least some embodiments, the method detects a vehicle reaction by, for example, Figure 1 The vehicle 140 or Figure 4 The detection vehicle system 410 is shown. In at least some embodiments, the method is performed by Figure 4 The detection components of the controller such as the ECU420 shown or Figure 8 The controller 802 is shown to perform.

[0088] In S660, the detection component or a subcomponent thereof determines whether the available storage has decreased. In at least some embodiments, the detection component determines whether the amount of available storage has decreased. In response to the detection component determining that the amount of available storage has not decreased, the action flow proceeds to a determination related to an increase in available storage in S661. In response to the detection component determining that the amount of available storage has decreased, the action flow proceeds to an increase in a threshold value of a triggering sensor in S664.

[0089] In S661, the detection component or a subcomponent thereof determines whether the available storage has increased. In at least some embodiments, the detection component determines whether the amount of available storage has increased. In response to the detection component determining that the amount of available storage has not increased, the action flow proceeds to a determination in S666 regarding whether a read threshold for a trigger sensor has been reached. In response to the detection component determining that the amount of available storage has increased, the action flow proceeds to a decrease in the threshold for a trigger sensor in S662.

[0090] In S662, the detection component or its subcomponent reduces the threshold of the trigger sensor. In at least some embodiments, the detection component reduces the reading threshold of at least one trigger sensor. In at least some embodiments, the detection component reduces the reading threshold of each trigger sensor by a small amount of the detection range of the trigger sensor.

[0091] In S664, the detection component or its subcomponent increases the threshold of the trigger sensor. In at least some embodiments, the detection component increases the read threshold of at least one trigger sensor. In at least some embodiments, the detection component increases the read threshold of each trigger sensor by a small amount of the detection range of the trigger sensor.

[0092] In S666, the detection component or a subcomponent thereof determines whether a read threshold of the trigger sensor has been reached. In at least some embodiments, the detection component determines whether a read threshold of at least one trigger sensor of the vehicle has been reached or exceeded. In response to not reaching the read threshold of the trigger sensor, the action flow returns to the determination related to the reduction of available storage in S660. In response to reaching the read threshold of the trigger sensor, the action flow ends and enters Figure 5 Action S552 records the output of sensors.

[0093] Figure 7 The process flow of recording vehicle reactions of at least some embodiments of the subject disclosure provides a method for recording detected vehicle reactions. In at least some embodiments, the method is performed by a vehicle, such as Figure 1 The vehicle 140 or Figure 4 The detection vehicle system 410 is shown. In at least some embodiments, the method is performed by Figure 4 The recording part of the controller such as ECU420 shown or Figure 8 The controller 802 is shown to perform.

[0094] In S770, the recording component or its subcomponent confirms whether the vehicle reaction is detected. In response to the vehicle reaction being detected, the action flow enters the recording of the output to the storage in S772. In response to the vehicle reaction not being detected, the action flow ends.

[0095] In S772, the recording component or its subcomponent records the output in a storage device. In at least some embodiments, the recording component records the output of at least one trigger sensor and at least one associated sensor of the vehicle in response to the detection of the vehicle reaction. In at least some embodiments, the recording component records the output of at least one trigger sensor and at least one associated sensor in a storage device. In at least some embodiments, the ECU that monitors such a sensor is programmed to perform an additional task for recording the output of the sensor, associated sensors, etc.

[0096] In S774, the recording component or its subcomponent determines whether the buffer record is available. In at least some embodiments, the recording component determines whether the buffer memory record of the sensor is available. In at least some embodiments, the buffer memory record always holds the records of all sensors a few seconds / minutes ago. In response to the buffer record being available, the action flow proceeds to copying the buffer record in S776. In response to the buffer record being unavailable, the action flow ends.

[0097] At S776, the recording component or its subcomponent copies the buffered record. In at least some embodiments, the recording component copies the buffered record of at least one triggering sensor and at least one associated sensor to a storage. In at least some embodiments, in the event of a trigger, the buffered memory record of the associated sensor is copied to the storage to constitute the beginning of the recording event, so that the moment before the trigger is available.

[0098] Figure 8 1 is a diagram of hardware configuration associated with determining ground object characteristics based on recording of vehicle events in accordance with at least some embodiments of the subject disclosure.

[0099] The preferred hardware configuration includes a detection vehicle system 800 that interacts with an input device 809, the detection vehicle system 800 communicating with the input device 809 and a server 810 via a network 808. In at least some embodiments, the detection vehicle system 800 is a computer or other computing device that receives input or commands from the input device 809 and the server 810. In at least some embodiments, the detection vehicle system 800 is embedded in the input device 809. In at least some embodiments, the detection vehicle system 800 is a computer system that executes computer-readable instructions in a manner that performs actions related to determining ground target characteristics based on vehicle records.

[0100] The detection vehicle system 800 includes a controller 802, a storage unit 804, an input / output interface 806, and a communication interface 807. In at least some embodiments, the controller 802 includes a processor or a programmable circuit that executes instructions, and the controller 802 causes the processor or the programmable circuit to act according to the instructions. In at least some embodiments, the controller 802 includes an analog or digital programmable circuit or any combination thereof. In at least some embodiments, the controller 802 includes a physically separated storage or circuit that interacts through communication. In at least some embodiments, the storage unit 804 includes a non-volatile computer-readable medium that can store executable data and non-executable data accessed by the controller 802 in the execution of instructions. The communication interface 807 sends and receives data with the network 808. The input / output interface 806 is connected to various input units and output units such as the input device 809 via a parallel port, a serial port, a keyboard port, a mouse port, a monitor port, and similar ports to accept commands and present information. In some embodiments, the storage unit 804 is located outside the server 800.

[0101] The controller 802 includes a detection component 820, a recording component 822, a determination component 824, and a transmission component 826. The storage unit 804 includes an output 830 of a sensor, a buffer record 832, a target characteristic 834, and a vehicle characteristic 836.

[0102] The detection component 820 is a circuit or instruction of the controller 802 configured to detect a vehicle reaction. In at least some embodiments, the detection component 820 is configured to detect a vehicle reaction of the vehicle, and the vehicle reaction is triggered by a reading threshold of at least one trigger sensor of the vehicle. In at least some embodiments, the detection component 820 uses information in the storage unit 804, such as the output 830 of the sensor. In at least some embodiments, as described in the above flowchart, the detection component 820 includes sub-components that perform additional functions. In at least some embodiments, such sub-components are referenced by names associated with corresponding functions.

[0103] Recording component 822 is a circuit or instruction of controller 802 configured to record output of a sensor. In at least some embodiments, recording component 822 is configured to record output of at least one trigger sensor and at least one associated sensor of the vehicle in response to detection of a vehicle reaction. In at least some embodiments, recording component 822 utilizes information in storage unit 804 such as sensor output 830 to record information such as buffer record 832 in storage unit 804. In at least some embodiments, as described in the above flowchart, recording component 822 includes subcomponents that perform additional functions. In at least some embodiments, such subcomponents are referenced by names associated with corresponding functions.

[0104] Determining component 824 is a circuit or instruction of controller 802 configured to determine a target characteristic of the ground. In at least some embodiments, determining component 824 is configured to determine the target characteristic of the ground based on the output of a record associated with at least one trigger sensor, the output of a record associated with at least one associated sensor, and at least one characteristic of the vehicle. In at least some embodiments, determining component 824 uses information in storage unit 804 such as sensor output 830, buffer record 832, and vehicle characteristic 836 to record information such as target characteristic 834 in storage unit 804. In at least some embodiments, as described in the above flowchart, determining component 824 includes subcomponents that perform additional functions. In at least some embodiments, such subcomponents are referenced by names associated with corresponding functions.

[0105] The sending component 826 is a circuit or instruction of the controller 802 configured to send the target characteristic. In at least some embodiments, the sending component 826 is configured to send the target characteristic to a server via a network. In at least some embodiments, the sending component 826 is configured to send the target characteristic directly to another vehicle. In at least some embodiments, the sending component 826 sends information such as the target characteristic 834 from the storage unit 804 to the network 808. In at least some embodiments, as described in the above flowchart, the sending component 826 includes subcomponents that perform additional functions. In at least some embodiments, such subcomponents are referenced by names associated with corresponding functions.

[0106] In at least some embodiments, the device is another device that can process logical functions to perform the actions of this specification. In at least some embodiments, the controller and the storage unit do not have to be completely separate devices. In some embodiments, the controller and the storage unit share circuits or one or more computer-readable media. In at least some embodiments, the storage unit includes a hard disk drive that stores both computer-executable instructions and data accessed by the controller, and the controller includes a combination of a central processing unit (CPU) and a RAM (Random Access Memory), where the computer-executable instructions can be copied in whole or in part in a manner that is executed by the CPU in the execution of the actions of this specification.

[0107] In at least some embodiments where the device is a computer, a program installed on the computer can cause the computer to function or perform actions associated with the device of the embodiments described in this specification. In at least some embodiments, such a program can be executed by a processor in a manner that causes the computer to perform specific actions associated with part or all of the functional blocks and block diagrams of the flowcharts described in this specification.

[0108] At least some embodiments are described with reference to flow charts and block diagrams, whose functional blocks represent (1) steps of a process for performing an action, or (2) components of a controller responsible for executing an action. In at least some embodiments, specific steps and components are implemented by dedicated circuits, programmable circuits supplied with computer-readable instructions stored on a computer-readable medium (or a computer program product), and / or processors supplied with computer-readable instructions stored on a computer-readable medium (or a computer program product). In at least some embodiments, the dedicated circuits include digital and / or analog hardware circuits, and include integrated circuits (ICs) and / or discrete circuits. In at least some embodiments, the programmable circuits include reconfigurable hardware circuits with logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, memory elements, etc., such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc.

[0109] In at least some embodiments, the computer-readable storage medium includes a tangible device that can hold and store instructions used by the instruction execution device. In some embodiments, the computer-readable storage medium includes, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above, but is not limited to these. A non-exhaustive list of more specific examples of computer-readable storage media includes the following, namely, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD (Compact Disc) – ROM), a digital versatile disc (DVD: Digital Versatile Disc), a memory stick (memory stick), a floppy disk, a punch card with instructions recorded or a mechanically encoded device such as a raised structure in a groove, and any suitable combination of the above. The computer-readable storage medium used in this specification should not be interpreted as temporary signals themselves such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses passing through fiber optic cables), or electrical signals sent through wires.

[0110] In at least some embodiments, the computer-readable program instructions described in this specification can be downloaded from a computer-readable storage medium to a respective computing / processing device, or can be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. In at least some embodiments, the network includes copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. In at least some embodiments, a network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and transmits the computer-readable program instructions to be stored in a computer-readable storage medium in the respective computing / processing device.

[0111] In at least some embodiments, the computer-readable program instructions for performing the above actions are any of assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source code or object code, the source code or object code being written in any combination of one or more programming languages, wherein the programming languages ​​include object-oriented programming languages ​​such as Smalltalk, C++, or similar programming languages, and conventional procedural programming languages ​​such as "C" programming languages ​​or similar programming languages, or modern system languages ​​such as Rust, or interface languages ​​such as Dart. In at least some embodiments, the computer-readable program instructions are executed entirely on the user's computer, or partially on the user's computer, or as a stand-alone software package, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In at least some embodiments, in the latter case, the remote computer is connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or (for example, using an Internet service provider to connect to an external computer via the Internet). In at least some embodiments, in order to perform the disclosed scheme of the subject matter, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), uses state information of computer-readable program instructions to individualize the electronic circuit, thereby executing the computer-readable program instructions.

[0112] Although the disclosed embodiments of the subject matter are described, the technical scope of any subject matter for which protection is sought is not limited to the above-mentioned embodiments. It should be understood by those skilled in the art that various changes and improvements to the above-mentioned embodiments can be made. It should also be understood by those skilled in the art from the claims that the embodiments to which such changes or improvements are added are included within the technical scope of the present invention.

[0113] The actions, procedures, steps, and platforms of each process performed by the apparatus, system, program, and method described in the claims, embodiments, or drawings may be performed in any order, as long as the order is not indicated by "first," "before," or similar terms, and as long as the output from the previous process is not used in the subsequent process. Even if the process flow is described in the claims, embodiments, or drawings using phrases such as "first" or "next," such description does not necessarily mean that the processes must be performed in the described order.

[0114] In at least some embodiments, determination of ground target characteristics based on vehicle records is performed by: detecting a vehicle reaction of the vehicle, wherein the vehicle reaction is triggered by a reading threshold of at least one trigger sensor of the vehicle; recording outputs of at least one trigger sensor and at least one associated sensor of the vehicle in response to detection of the vehicle reaction; and determining the ground target characteristics based on the recorded output associated with the at least one trigger sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle.

[0115] Some embodiments include instructions in a computer program, methods performed by a processor executing the instructions of the computer program, and apparatus for performing the methods. In some embodiments, the apparatus includes a controller including circuits configured to perform the actions in the instructions.

[0116] The features of several embodiments are summarized above in a manner that those skilled in the art can more fully understand the scheme of the present disclosure. It should be understood by those skilled in the art that they can easily use the present disclosure as a basis for designing or modifying other processes and structures, wherein the other processes and structures perform the same purposes and / or achieve the same advantages as the embodiments written in this specification. It should also be understood by those skilled in the art that such equivalent structures do not depart from the subject matter and scope of the present disclosure, and that those skilled in the art can make various changes, substitutions, and modifications to the present specification without departing from the subject matter and scope of the present disclosure.

Claims

1. A method for determining characteristics of a ground target, comprising: detecting a vehicle reaction of a vehicle, wherein the vehicle reaction is triggered by a reading threshold of at least one triggering sensor of the vehicle; recording outputs of the at least one triggering sensor and at least one associated sensor of the vehicle in response to detection of the vehicle reaction; and A target characteristic of the ground is determined based on the recorded output associated with the at least one triggering sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle.

2. The method for determining the characteristics of a ground target according to claim 1, wherein: The records include: recording the output of the at least one trigger sensor and the output of the at least one associated sensor in a memory; and A buffer record of the at least one triggering sensor and the at least one associated sensor is copied to the storage.

3. The method for determining the characteristics of a ground target according to claim 1 or 2, further comprising: The target characteristics are sent to a server via a network.

4. The method for determining the characteristics of a ground target according to any one of claims 1 to 3, further comprising: The target characteristic is sent directly to another vehicle.

5. The method for determining the characteristics of a ground target according to any one of claims 1 to 4, further comprising: The read threshold is adjusted based on the amount of available storage.

6. The method for determining the characteristics of a ground target according to claim 5, wherein: The adjustment also includes at least one of the following: decreasing the read threshold in response to an increase in the amount of available storage; or The read threshold is increased in response to a decrease in the amount of available storage.

7. The method for determining the characteristics of a ground target according to any one of claims 1 to 6, wherein: The at least one associated sensor includes a GPS sensor.

8. The method for determining the characteristics of a ground target according to any one of claims 1 to 7, wherein: The at least one characteristic of the vehicle includes at least one of a static characteristic based on a make and model of the vehicle or a dynamic characteristic of the vehicle during the recorded time.

9. The method for determining the characteristics of a ground target according to any one of claims 1 to 8, wherein: The read threshold is based on the operational design domain ODD.

10. A computer program product having recorded thereon instructions, wherein the instructions can be executed by one or more processors to perform actions, the actions comprising: detecting a vehicle reaction of a vehicle, wherein the vehicle reaction is triggered by a reading threshold of at least one triggering sensor of the vehicle; recording outputs of the at least one triggering sensor and at least one associated sensor of the vehicle in response to detection of the vehicle reaction; and A target characteristic of the ground is determined based on the recorded output associated with the at least one triggering sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle.

11. The computer program product according to claim 10, wherein: The records include: recording the output of the at least one trigger sensor and the output of the at least one associated sensor in a memory; and A buffer record of the at least one triggering sensor and the at least one associated sensor is copied to the storage.

12. The computer program product according to claim 10 or 11, wherein: The actions also include: sending the target characteristic to a server via a network.

13. The computer program product according to any one of claims 10 to 12, wherein: The actions also include sending the target characteristic directly to another vehicle.

14. The computer program product according to any one of claims 10 to 13, wherein: The actions also include adjusting the read threshold based on an amount of available storage.

15. The computer program product of claim 14, wherein: The adjustment also includes at least one of the following: decreasing the read threshold in response to an increase in the amount of available storage; or The read threshold is increased in response to a decrease in the amount of available storage.

16. The computer program product according to any one of claims 10 to 15, wherein: The at least one associated sensor includes a GPS sensor.

17. The computer program product according to any one of claims 10 to 16, wherein: The at least one characteristic of the vehicle includes at least one of a static characteristic based on a make and model of the vehicle or a dynamic characteristic of the vehicle during the recorded time.

18. The computer program product according to any one of claims 10 to 17, wherein: The read threshold is based on the operational design domain ODD.

19. An apparatus for determining characteristics of a ground target, comprising a controller including circuitry configured to: Detecting a vehicle reaction of a vehicle, wherein The vehicle reaction is triggered by a reading threshold of at least one triggering sensor of the vehicle; recording outputs of the at least one triggering sensor and at least one associated sensor of the vehicle in response to detection of a reaction of the vehicle; as well as A target characteristic of the ground is determined based on the recorded output associated with the at least one triggering sensor, the recorded output associated with the at least one associated sensor, and at least one characteristic of the vehicle.

20. The device for determining the characteristics of a ground target according to claim 19, wherein: The circuit configured to record is configured to: recording the output of the at least one trigger sensor and the output of the at least one associated sensor in a memory; as well as A buffer record of the at least one triggering sensor and the at least one associated sensor is copied to the storage.

Images