Road type identification method and device, vehicle and computer readable storage medium

Abstract

The invention relates to the field of recognition, in particular to a road type recognition method and device, a vehicle and a computer readable storage medium. The method comprises the following steps: acquiring a current driving state parameter of a vehicle and a front road image of the vehicle at target time; the target time is determined according to the vehicle speed and the collection position difference; the acquisition position difference represents the distance between a first acquisition position of the front road image and a second acquisition position of the driving state parameter; identifying the front road image to obtain a first prediction result of the road type of the road where the vehicle is located currently; determining a second prediction result of the road type according to the driving state parameters; and determining the target type of the current road of the vehicle according to the first prediction result and the second prediction result. According to the invention, the road type identification accuracy of the road where the vehicle is located can be improved.

Description

Technical Field

[0001] The present application relates to the field of recognition technologies, and particularly relates to a road type recognition method, device, vehicle, and computer-readable storage medium. Background Art

[0002] The recognition of the road type of the road where a vehicle is located can provide a reference for the control strategy of a specified vehicle. In related technologies, the road type of the road where the vehicle is located can be recognized by collecting one or more modal data of the vehicle's environment and based on the collected data.

[0003] When determining the road type of the vehicle based on multi-modal data, related technologies generally directly compare and fuse different modal data collected, which has the problem of low accuracy in road type recognition, affecting the reliability of vehicle control and the driving experience. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a road type recognition method, device, vehicle, and computer-readable storage medium, which can solve the problem of low accuracy in existing road type recognition.

[0005] In a first aspect, an embodiment of the present application provides a road type recognition method, which includes: Obtain the current driving state parameters of the vehicle and the front road image of the vehicle at a target time; the target time is determined according to the vehicle speed and the acquisition position difference; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameters; Recognize the front road image to obtain a first prediction result of the road type of the road where the vehicle is currently located; Determine a second prediction result of the road type according to the driving state parameters; Determine the target type of the road where the vehicle is currently located according to the first prediction result and the second prediction result.

[0006] With the above technical solution, based on the acquisition characteristics of the front road image of the vehicle and the differences in the acquisition methods between the front road image and the driving state parameters, the road area characterized by the driving state parameters is generally the road where the vehicle is currently located or has just passed, and it has a greater representativeness of the road type of the current road. While the road area characterized by the front road image is generally within a certain range in front of the vehicle, and there is a certain position difference between it and the location of the current road. Therefore, different from the related art where the acquisition methods of different types of data, such as the difference in acquisition positions, are not considered, and the recognition results of the front road image and the driving state parameters collected at the same moment are directly compared, resulting in a relatively large recognition error, the embodiments of the present invention take into account that the vehicle will reach the position characterized by the front road image collected at its historical moment during driving, and the arrival time is related to the vehicle speed and the acquisition position difference between the front road image and the driving state parameters. Therefore, according to the vehicle speed and the acquisition position difference, the front road image collected at the target time in history that can represent the position of the vehicle at the current time is determined, so as to ensure that the visual data and the driving state data used as the basis for road type recognition correspond to the same position where the vehicle is located. The embodiments of the present invention take into account the acquisition position difference between the front road image and the driving state parameters of the vehicle and the motion characteristics of the vehicle during the acquisition process, and more accurately identify the road type of the vehicle, providing a reliable reference for subsequent working conditions such as vehicle control.

[0007] In some embodiments, the target time is one of a plurality of preset candidate times; the vehicle speed includes the speeds of the vehicle at each of the candidate times; the plurality of preset candidate times are earlier than the current time; the method further includes: For each of the candidate times, according to the speed corresponding to each of the candidate times and the acquisition position difference, determine the actual arrival time of the vehicle at the road area characterized by the front road image collected at each candidate time; Match the actual arrival time corresponding to each candidate time with the current time, and use the candidate time corresponding to the actual arrival time that matches the current time as the target time.

[0008] In some embodiments, the first prediction result includes the first confidence level that the vehicle is currently in a preset road type; the method further includes: Identify the front road image to obtain the original confidence level that the vehicle is currently in the preset road type; Analyze the driving environment of the vehicle to obtain the influence degree of the driving environment on the recognition accuracy of the front road image; The original confidence level is corrected according to the influence degree to obtain the first confidence level; wherein, the influence degree is negatively correlated with the first confidence level.

[0009] In some embodiments, the method further includes: Analyze the driving environment of the vehicle according to preset harsh environment characteristics to obtain the harshness degree of the driving environment; the harsh environment characteristics include environment characteristics that affect the visibility of the driving environment; Determine the influence degree according to the harshness degree; wherein, the harshness degree is positively correlated with the influence degree.

[0010] In some embodiments, the first prediction result includes the first confidence level of the vehicle currently being in multiple preset road types; the second prediction result includes the second confidence level of the vehicle currently being in the multiple preset road types; the method further includes: Compare the first confidence levels corresponding to each of the preset road types, and determine the preset road type with the maximum first confidence level as the first candidate type; Compare the second confidence levels corresponding to each of the preset road types, and determine the preset road type with the maximum second confidence level as the second candidate type; Determine the target type according to the first candidate type, the second candidate type, and a preset confidence level threshold.

[0011] In some embodiments, the driving state parameters include the chassis state parameters and / or power state parameters of the vehicle; the correlation between the chassis state parameters and / or power state parameters and the control operations of the driver on the vehicle is less than a preset threshold.

[0012] In some embodiments, the chassis state parameters and / or power state parameters include at least one of the four-wheel wheel speed signals, vehicle speed signal, braking torque, steering angle, steering angular velocity, longitudinal acceleration, lateral acceleration, yaw angular velocity, total vehicle driving torque, and four-wheel suspension height of the vehicle.

[0013] In a second aspect, an embodiment of the present application further provides a road type recognition device, including: An acquisition module, configured to acquire the current driving state parameters of the vehicle and the front road image of the vehicle at a target time; the target time is determined according to the vehicle speed and the acquisition position difference of the vehicle; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameters; An identification module, configured to identify the front road image to obtain a first prediction result of the road type of the road where the vehicle is currently located; A first determination module, configured to determine a second prediction result of the road type according to the driving state parameters; A second determination module, configured to determine the target type of the road where the vehicle is currently located according to the first prediction result and the second prediction result.

[0014] In a third aspect, an embodiment of the present application further provides a vehicle, including: a processor and a memory, where the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the vehicle executes the road type identification method as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and when the computer instructions run on a processor, the processor is caused to execute the road type identification method as described in the first aspect. Description of the Drawings

[0016] Figure 1 It is a flowchart of steps of a road type identification method according to an embodiment of the present application.

[0017] Figure 2 It is a flowchart of steps of a road type identification method according to another embodiment of the present application.

[0018] Figure 3 It is a flowchart of steps of a road type identification method according to another embodiment of the present application.

[0019] Figure 4 It is a flowchart of steps of a road type identification method according to another embodiment of the present application.

[0020] Figure 5 It is a flowchart of steps of a road type identification method according to another embodiment of the present application.

[0021] Figure 6 It is a schematic structural diagram of a road type identification device according to an embodiment of the present application.

[0022] Figure 7 It is a schematic structural diagram of a vehicle according to an embodiment of the present application. Detailed Embodiments

[0023] In order to more clearly understand the above objects, features and advantages of the present application, the present application will be described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.

[0024] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. The described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments, and are not intended to limit this application.

[0026] Further, it should be noted that in this document, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element.

[0027] In this application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the relationship between associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, or B exists alone, where A and B may be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0028] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0029] In the related art, for the needs of vehicle control and the like, the road type of the vehicle can be identified. The methods for road type identification can include: visually identifying the environment where the vehicle is located, identifying the dynamic state of the vehicle, etc. In order to improve the accuracy of road type identification, multiple modal data can be fused for road type identification.

[0030] In the related art, generally, data of multiple modalities collected at the same moment are directly analyzed, and the road type is determined according to the analysis results. It does not consider that there are differences in the acquisition methods of data of different modalities, such as the different setting positions of the data acquisition devices, different acquisition ranges, etc. For example, when identifying the road type based on the front road image and the driving state of the vehicle, due to the motion characteristics of the vehicle and the setting position of the front camera, there is a certain deviation between the road position represented by the front road image and the road position represented by the driving state. Therefore, regarding the front road image and the driving state as representing the same position in the related art will result in a low accuracy of road type recognition.

[0031] In view of this, the embodiments of the present application provide a road type recognition method, device, equipment and computer-readable storage medium, which can improve the accuracy of road type recognition of the vehicle.

[0032] Please refer to Figure 1 , which is a step flowchart of the road type recognition method provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The embodiments of the present invention are executed based on a preset electronic device, which has certain communication capabilities, data processing capabilities and data storage capabilities. Specifically, the electronic device can be an in-vehicle controller, etc. The electronic device establishes a communication connection with preset sensors on the vehicle. The preset sensors can include the front camera, radar, chassis sensors, etc. of the vehicle. The embodiments of the present invention do not limit this.

[0033] Refer to Figure 1 As shown, the road recognition method may include the following steps: Step 101: Obtain the current driving state parameters of the vehicle and the front road image of the vehicle at the target time; the target time is determined according to the vehicle speed and the acquisition position difference; the target time is determined according to the vehicle speed and the acquisition position difference; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameters.

[0034] Among them, the driving state parameters are used to characterize the dynamic response of the vehicle to the road. It can be understood that different types of roads have different effects on the vehicle. For example, when driving on roads with different terrains such as asphalt roads, cement roads, grasslands, and muddy roads, the response parameters of the vehicle's chassis system, power system, etc. to this effect also change accordingly. Therefore, the road type where the vehicle is currently located can be inferred by obtaining the current driving state parameters of the vehicle. The driving state parameters can be collected by preset sensors set on the vehicle's chassis system and power system, such as wheel speed sensors, torque sensors, etc. It can be understood that the collection range targeted by the preset sensors set on the vehicle's chassis system and power system generally includes the position where the vehicle is currently located or has just passed through. Therefore, the timeliness of the characterization result of the driving state parameters for the road where the vehicle is located is relatively strong, and the road type where the vehicle is currently located can be determined according to the current driving state parameters.

[0035] Correspondingly, based on the setting characteristics of the vision sensors on the vehicle and the movement characteristics of the vehicle, in related technologies, the road image of the road in front of the vehicle is generally obtained through the front camera on the vehicle as the image of the road where the vehicle is currently located. Since the road image in front represents the position that the vehicle will travel to within a certain period of time in the future, and there is a certain distance between this position and the position where the vehicle is currently located, in order to more accurately predict the road type of the position where the vehicle is currently located, it is necessary to use the road image in front collected in advance before the current time as the image of the road where the vehicle is located at the current time, so as to predict the road type based on this image.

[0036] Among them, in order to more accurately screen out the image that can represent the position where the vehicle is currently located, in the embodiments of the present invention, according to the ratio of the distance between the first collection position of the road image in front and the second collection position of the driving state parameters to the vehicle speed, the advance duration of the collection time of the road image in front representing the current position relative to the current time is determined. According to this advance duration, moving forward from the current time, the target time is obtained. Thus, the road image in front collected at the target time can represent the image of the road where the vehicle is located at the current time.

[0037] Among them, the first collection position represents the position of the collection area of the road image in front, such as an area at a preset distance in front of the vehicle. Among them, the preset distance can be 5 meters to 10 meters, which is determined according to the collection range parameters of the collection device of the road image in front. Correspondingly, the second collection position represents the position of the collection area of the driving state parameters. Considering that the response speed of the vehicle's driving state to terrain changes is relatively fast, therefore, the second collection position of the driving state parameters at the current time can be regarded as the position where the vehicle is currently located.

[0038] For example, for the current time, the second acquisition position may be the position of the road mapped by the center point between the front wheels of the current vehicle, and the first acquisition position may be the position of the road acquired by the front camera of the vehicle at the current time, such as the position 5 meters in front of the vehicle. Considering the difference between the setting positions of the front wheels and the front camera of the vehicle, the acquisition position difference may be 6 meters. The vehicle speed of the vehicle may be the speed of the vehicle at the current time. Optionally, considering that the position of the vehicle is dynamically changing during driving, therefore, the vehicle speed may be the vehicle speeds at various historical times before the current time, and the target time is one of the multiple historical times.

[0039] Optionally, the control process of the driver on the vehicle is described as follows: The driver or the driverless system or the assisted driving system inputs operation actions or operation instructions to the vehicle. The vehicle generates actual operation responses and sends out relevant operation signals and status signals. The driving preference analysis module sends out driving preference signals according to the operation actions or operation instructions of the driver. The electronic control suspension roll control module receives the vehicle status signal / operation signal, driving preference signal, and the feedback signal of the electronic control suspension actuator module, calculates the control force required for roll control, and sends the control instruction to the electronic control suspension actuator module. The electronic control suspension execution module converts the control instruction into a control force and applies it to the vehicle suspension system, thereby changing the vehicle state, improving the comfort or handling stability of the vehicle, and at the same time feeding back the state of the actuator itself to the electronic control suspension roll control module. In the electronic control suspension roll control module, first, the vehicle state is estimated, second, the total control force is calculated, second, the control force after the front axle is distributed, then the control force is compensated, then the control force is corrected, and finally the control force is output.

[0040] Based on the above vehicle control process, considering that the driving state parameters of some vehicles may be greatly affected by driving control operations, such as vehicle controllers such as drivers or assisted driving systems can control the torque, braking force, etc. of the vehicle. Considering that vehicle controllers generally adjust vehicle control based on the judgment of the current road type, such as using an off-road mode on an ice and snow road and adopting a driving mode corresponding to a conventional public road on an asphalt or cement road. However, there may be a certain time interval between when the vehicle controller judges the road terrain and when it adjusts vehicle driving control according to the judgment result, resulting in the inability to timely switch the driving control parameters to parameters suitable for the current road terrain type, thus causing the driving state of the vehicle after being controlled by driving not to match the actual road type where the vehicle is located.

[0041] Therefore, in some embodiments, in order to improve the accuracy of estimating the type of the road of the vehicle based on the driving state parameters, parameters that are less affected by the driver's driving control operations and can characterize the dynamic response of the vehicle to the road can be selected. Specifically, the driving state parameters include the chassis state parameters and / or the power state parameters of the vehicle; the correlation between the chassis state parameters and / or the power state parameters and the driver's control operations on the vehicle is less than a preset threshold.

[0042] Among them, considering that the components mainly interacting between the vehicle and the road are generally the chassis domain and / or the power domain, therefore, the driving state parameters include the chassis state parameters and / or the power state parameters of the vehicle, where the chassis state parameters characterize the state of the chassis domain of the vehicle, and the power state parameters characterize the state of the power domain of the vehicle.

[0043] The driver, who is the object of driving control of the vehicle, can be a driver, or an assisted driving system, an autonomous driving system, etc., and the present invention does not limit this. The driver can affect the driving state parameters of the vehicle by performing driving control on the vehicle, such as selecting and adjusting the driving mode. The correlation is used to characterize the influence degree of the driver's control operations on the vehicle on the chassis state parameters and / or the power state parameters of the vehicle. Considering that when the driver drives the vehicle on different types of road surfaces in the same way, the performance differences of the parameters with less correlation with the driving operations will be larger, which can be used to distinguish the current road. Correspondingly, due to the influence of subjective control behaviors, the parameters with a large correlation with the driving control have a relatively low representation degree for the objective road type. Therefore, in the embodiments of the present invention, the chassis state parameters and / or the power state parameters with a correlation less than a preset threshold with the driver's control operations on the vehicle are selected as the driving state parameters.

[0044] Optionally, by measuring the driving state parameters of the vehicle under different road types and different driving operations, it is obtained that vehicle parameters such as the four-wheel wheel speed signal, vehicle speed signal, braking torque, steering angle, steering angular velocity, longitudinal acceleration, lateral acceleration, yaw angular velocity, vehicle driving torque, and four-wheel suspension height of the vehicle are less affected by the driver's driving operations. Therefore, in some embodiments, the chassis state parameters and / or the power state parameters include at least one of the four-wheel wheel speed signal, vehicle speed signal, braking torque, steering angle, steering angular velocity, longitudinal acceleration, lateral acceleration, yaw angular velocity, vehicle driving torque, and four-wheel suspension height of the vehicle.

[0045] It can be understood that considering that drivers generally have commonalities in controlling vehicles based on the current terrain, in some embodiments of the present invention, weights can also be set for different driving state parameters. The selected weights are used to represent the probability that the driving state parameters are used to determine the road type of the vehicle, and the selected weights are positively correlated with the degree of influence of the parameters on driving operations.

[0046] Step 102: Identify the front road image to obtain a first prediction result of the current road type of the vehicle.

[0047] Among them, the front road image can be identified through a preset image recognition model to obtain the confidence levels of the vehicle being in each preset road type as the first prediction result. The preset road types can include asphalt roads, cement roads, dirt roads, grasslands, gravel roads, sandy lands, muddy lands, snow lands, ice surfaces, etc. The preset road types can be set and calibrated according to the road scenarios that the vehicle may be in under normal and / or extreme conditions, and the embodiments of the present invention do not limit this. The image recognition model can be trained based on a preset machine learning algorithm such as a convolutional neural network algorithm. The image recognition model can be trained with image samples as inputs and the corresponding road type labels of the image samples as outputs.

[0048] Step 103: Determine a second prediction result of the road type according to the driving state parameters.

[0049] Among them, the current driving state parameters of the vehicle can be identified through a preset state parameter recognition model to obtain the confidence levels of the vehicle being in each preset road type as the second prediction result. The state parameter recognition model can be trained based on a preset machine learning algorithm. Specifically, the state parameter recognition model can be based on, for example, SVM (Support Vector Machines), KNN (K-Nearest Neighbor), and random forests, etc., and the embodiments of the present invention do not limit this. The state parameter recognition model can be trained with state parameter samples as inputs and the corresponding road type labels of the state parameter samples as outputs.

[0050] Step 104: Determine the target type of the current road of the vehicle according to the first prediction result and the second prediction result.

[0051] Among them, the forward road image can be recognized by a preset image recognition model to obtain the confidence of the vehicle currently in each preset road type as the first prediction result. The current driving state parameters of the vehicle can be recognized by a preset state parameter recognition model to obtain the confidence of the vehicle currently in each preset road type as the second prediction result. The state parameter recognition model can be trained based on a preset machine learning algorithm. Then, the road type with the highest confidence is selected as the target type according to the first prediction result and the second prediction result.

[0052] Optionally, after determining the target type, the control strategy for the vehicle can also be adjusted according to the target type. For example, the driving mode of the vehicle can be switched to the preset driving mode corresponding to the target type, so as to better ensure driving safety and riding experience.

[0053] In some embodiments, considering the limitations of the acquisition device and the displacement characteristics during vehicle driving, the road image of the vehicle collected is generally the forward road image of the vehicle, and the forward road image represents the position that the vehicle will drive to in the future, while the timeliness of the driving state parameters is generally strong, which can represent the road conditions of the road where the vehicle is currently located or has just passed. Therefore, there is a position difference between the driving state parameters collected at the same time and the road area represented by the forward road image of the vehicle. Therefore, in order to improve the determination of the road type where the vehicle is located at a specific moment based on the driving state parameters and the forward road image, such as Figure 2 shown, the process of road type recognition includes: Step 201: Obtain the forward road image of the vehicle collected at multiple preset candidate times and the speed of the vehicle at the candidate times; the multiple preset candidate times are earlier than the current time.

[0054] Among them, multiple preset candidate times are selected at multiple time points according to a preset frequency within the historical time interval within a preset duration before the current time. For example, the candidate time can be 0.2 seconds, 0.3 seconds, 0.4 seconds, 0.5 seconds, 1 second, 2 seconds, 4 seconds, 5 seconds, 8 seconds, 10 seconds, etc. before the current time. It can be understood that considering the time advance of the forward road image in representing the road conditions, the forward road image collected at the candidate time before the current time is obtained, so as to screen out the image that can represent the position of the vehicle at the current time from the forward road images collected in the historical time interval.

[0055] Considering the position that the vehicle will travel to within a certain future time based on the representation of the road image ahead, there is a certain distance between this position and the position where the vehicle is currently located. And with different vehicle speeds, the time it takes to travel to the position corresponding to the road image ahead collected at the candidate time is different. The faster the vehicle speed, the closer the target time is to the current time. Therefore, it is also necessary to obtain the speed of the vehicle at the candidate time.

[0056] Step 202: For each of the candidate times, determine the actual arrival time of the vehicle at the road area represented by the road image ahead collected at each candidate time according to the vehicle speed corresponding to the candidate time and the acquisition position difference; the acquisition position difference represents the distance between the first acquisition position of the road image ahead and the second acquisition position of the driving state parameter.

[0057] Among them, the determination process of the acquisition position difference can refer to the determination process in the aforementioned step 101 and will not be elaborated here. The real-time vehicle speed and the real-time road image ahead can be continuously collected during the vehicle driving, so that for the current time, the vehicle and the acquisition position difference at each candidate time before the current time can be obtained.

[0058] It should be noted that considering that as the vehicle travels, both the first acquisition position and the second acquisition position will change. However, since the installation positions and acquisition ranges of the acquisition device for the road image ahead and the acquisition device for the driving state are generally still fixed, during the vehicle driving, the acquisition position difference between the first acquisition position and the second acquisition position may change little. For example, it can be determined directly according to the acquisition range of the acquisition device for the road image ahead and the acquisition range of the driving state device. For the current time, the second acquisition position can be the position on the road mapped by the center point between the front wheels of the current vehicle, and the first acquisition position can be the road position collected by the front camera of the vehicle at the current time, such as the position 5 meters ahead of the vehicle, then the acquisition position difference may be 5 meters.

[0059] On this basis, considering that for the same acquisition position difference, when the vehicle speed is different, the time for the vehicle to actually reach the road area represented by the road image ahead collected at a specific candidate time also varies. Specifically, the faster the vehicle speed, the shorter the interval between the acquisition time of the road image ahead representing the road area where the vehicle is located at the current time (i.e., the road image ahead at the current time) and the current time.

[0060] Therefore, it is also necessary to obtain the vehicle speed at each candidate time, so as to more accurately screen out the acquisition time of the front road image at the position of the vehicle at the current time from multiple candidate times, and use this acquisition time as the target time, so that the front road image acquired at the target time corresponds to the position of the vehicle at the current time.

[0061] Specifically, for each candidate time, the ratio between the acquisition position difference corresponding to the candidate time and the vehicle speed at the candidate time can be determined as the actual arrival time of the vehicle at the road area represented by the front road image acquired at the candidate time.

[0062] For example, as shown in Table 1, assuming that the acquisition position difference corresponding to a certain candidate time is 6 meters, when the vehicle speed is different, the actual arrival time of the vehicle at the road area represented by the front road image acquired at the candidate time also varies: Table 1 Mapping relationship table between vehicle speed and actual arrival time Step 203: Match the actual arrival time corresponding to each candidate time with the current time, and use the candidate time corresponding to the actual arrival time that matches the current time as the target time.

[0063] Among them, the candidate time with the actual arrival time equal to or closest to the current time is determined as the target time, so that the front road image acquired at the target time corresponds to the position of the vehicle at the current time, thereby improving the prediction accuracy of the road type where the vehicle is located at the current time.

[0064] Step 204: Identify the front road image acquired at the target time to obtain a first prediction result of the road type where the vehicle is currently located.

[0065] Among them, the determination process of the first prediction result can refer to the aforementioned step 102, which will not be elaborated here.

[0066] Step 205: Obtain the current driving state parameters of the vehicle.

[0067] Among them, the acquisition process of the driving state parameters can refer to the aforementioned step 101, which will not be elaborated here.

[0068] Step 206: Determine a second prediction result of the road type according to the driving state parameters.

[0069] Among them, step 206 is substantially the same as the aforementioned step 103, which will not be elaborated here.

[0070] Step 207: Determine the target type of the current road of the vehicle according to the first prediction result and the second prediction result.

[0071] Among them, Step 207 is substantially the same as the foregoing Step 104 and will not be elaborated here.

[0072] In some embodiments, considering that the recognition accuracy of the front road image may be affected by interference factors in the acquisition environment such as weather and light. For example, for the front road images acquired in environments such as at night, in fog, or in rain, the reliability of the first prediction result identified therefrom will be relatively reduced. Therefore, in order to improve the recognition accuracy of the road type of the road where the vehicle is located, as Figure 3 shown, the road type recognition process includes: Step 301: Obtain the current driving state parameters of the vehicle and the front road image of the vehicle at the target time; the target time is determined according to the vehicle speed and the acquisition position difference; the target time is determined according to the vehicle speed and the acquisition position difference; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameters.

[0073] Among them, Step 301 is substantially the same as the foregoing Step 101 and will not be elaborated here.

[0074] Step 302: Recognize the front road image to obtain the original confidence level of the vehicle currently being in the preset road type.

[0075] Specifically, the front road image can be recognized by a preset image recognition model to obtain the original confidence levels of the vehicle currently being in each preset road type. The image recognition model can be trained based on a preset machine learning algorithm such as a convolutional neural network algorithm. The preset road types can include asphalt roads, cement roads, dirt roads, grasslands, gravel roads, sandy lands, muddy lands, snow lands, ice surfaces, etc. The preset road types can be set and calibrated according to the road scenarios that the vehicle may be in under normal and / or extreme conditions.

[0076] Step 303: Analyze the driving environment of the vehicle to obtain the influence degree of the driving environment on the recognition accuracy of the front road image.

[0077] Among them, considering that the recognition accuracy of the forward road image is affected by the driving environment of the vehicle. For example, when the visibility of the driving environment is low or there are other factors affecting the transmission and acquisition of visual features, the recognition accuracy of the forward road image will decrease, thereby affecting the reliability of the original confidence. Therefore, the driving environment of the vehicle can be detected according to the preset environmental features that affect the recognition accuracy, and the degree of influence of the driving environment on the recognition accuracy of the forward road image can be determined according to the detection results of the preset environmental features in the driving environment. The preset environmental features can be features such as rain, snow, fog, light intensity, etc. that have a direct or indirect impact on the acquisition and recognition of visual features. For example, when detecting the driving environment of the vehicle for the environmental feature of fog and obtaining that the fog concentration in the vehicle driving environment is greater than the preset concentration threshold, the degree of influence of the driving environment on the recognition accuracy of the forward road image is determined according to the content of the fog concentration.

[0078] Step 304: Modify the original confidence according to the degree of influence to obtain the first confidence that the vehicle is currently in the preset road type; wherein, the degree of influence is negatively correlated with the first confidence.

[0079] It can be understood that the greater the influence on the recognition accuracy of the visual recognition of the forward road image, the lower the reliability of the original confidence, that is, the lower the prediction accuracy for the actual road type of the vehicle. Therefore, the original confidence can be modified according to the degree of influence. When the degree of influence increases, the original confidence is correspondingly adjusted downward to obtain the first confidence that the vehicle is currently in the preset road type. This first confidence can eliminate the influence of environmental features such as heavy rain, night, thick fog, etc. that will cause large visual recognition errors, thereby improving the accuracy of predicting the road type of the vehicle based on the forward road image. For example, the first confidence = the original confidence × the degree of influence.

[0080] Step 305: Determine the second prediction result of the road type according to the driving state parameters.

[0081] Among them, step 305 is the same as the foregoing step 103 and will not be elaborated here.

[0082] Step 306: Determine the target type of the current road of the vehicle according to the first confidence that the vehicle is currently in the preset road type and the second prediction result.

[0083] Among them, step 306 is substantially the same as the foregoing step 104 and will not be elaborated here.

[0084] In some embodiments, considering that the reliability of the first prediction result obtained based on visual feature recognition is affected by the recognition accuracy of the front road image, and the recognition accuracy of the front road image collected in a low visibility environment is also low, thus affecting the reliability of the first prediction result. Therefore, as Figure 4 shown, the road type recognition process includes: Step 401: Obtain the current driving state parameters of the vehicle and the front road image of the vehicle at the target time; the target time is determined according to the vehicle speed and the acquisition position difference of the vehicle; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameters.

[0085] Among them, step 401 is substantially the same as the foregoing step 101, and will not be elaborated here.

[0086] Step 402: Recognize the front road image to obtain the original confidence level that the vehicle is currently in the preset road type.

[0087] Among them, step 402 is substantially the same as the foregoing step 302, and will not be elaborated here.

[0088] Step 403: Analyze the driving environment of the vehicle according to the preset harsh environment features to obtain the harsh degree of the driving environment; the harsh environment features include the environmental features that affect the visibility of the driving environment.

[0089] Among them, the harsh environment features include the features whose characteristic values in a specific feature dimension meet the preset threshold. Among them, considering the recognition situation of the visual features included in the front road image, when determining the road type that conforms to the visual features, the presentation, transmission, and acquisition of the visual features are all affected by the visibility of the driving environment of the vehicle. The lower the visibility, the lower the reliability of the presentation, transmission, and acquisition of the visual features. The reliability of predicting the road type based on the visual features with low reliability will also decrease accordingly. Therefore, this specific feature dimension represents the features that affect the visibility of the driving environment. The positivity or negativity and the degree of influence of this influence are determined according to the characteristic values in this feature dimension.

[0090] The severity level represents the matching degree of the driving environment of the vehicle to the characteristics of the harsh environment. The greater the severity level, the greater the degree to which the visibility in the driving environment is affected by the environment. For example, for a specific characteristic dimension such as rainfall, when the rainfall is greater than a preset threshold, it will have a greater impact on the accuracy of visual recognition. Therefore, heavy rain can be regarded as a harsh environment characteristic. Optionally, for a specific characteristic dimension such as fog concentration, when the fog concentration is greater than a preset threshold, it will have a greater impact on the accuracy of visual recognition. Therefore, thick fog can be regarded as a harsh environment characteristic. Optionally, for a specific characteristic dimension such as light intensity, when the light intensity is less than a certain level (such as when representing night time), it will have a greater impact on the accuracy of visual recognition. Therefore, when the light intensity is less than the preset threshold, it can be regarded as a harsh environment characteristic.

[0091] Step 404: Determine the degree of influence of the driving environment on the recognition accuracy of the front road image according to the severity level; wherein, the severity level is positively correlated with the degree of influence.

[0092] Among them, in order to incorporate the influence of the harsh environment into the road type prediction based on visual recognition in a more fine-grained manner, thereby improving the accuracy of road type prediction, for the severity level of the driving environment under the harsh environment characteristics in each dimension, determine the correction coefficient corresponding to the harsh environment characteristic of this dimension, and comprehensively determine the degree of influence according to the correction coefficients corresponding to the harsh environment characteristics of all dimensions.

[0093] For example, in order to eliminate the influence of harsh environments such as thick fog, heavy rain, and night that are not conducive to visual recognition on the confidence of visual recognition, the degree of influence can be calculated according to the following formula: Degree of influence = thick fog correction coefficient × rainfall correction coefficient × light intensity correction coefficient.

[0094] Among them, the thick fog correction coefficient is used to represent the severity level of the harsh environment characteristic of thick fog. The thick fog state has two states. When the thick fog flag bit is 1, the thick fog correction coefficient takes 0; when the thick fog flag bit is 0, the thick fog correction coefficient takes 1. Among them, the determination process of the thick fog flag bit includes: judging whether the vehicle is in thick fog when collecting the front road image by analyzing parameters such as the clarity, contrast, and color of the image. If so, output the thick fog flag bit 1, otherwise output the thick fog flag bit 0.

[0095] The rainfall correction coefficient is used to represent the severity level of the harsh environment characteristic of rainfall. The rainfall correction coefficient can be a constant between 0 and 1, and this constant can be obtained through pre-calibration. When it does not rain, the correction coefficient is 1. When it rains, the rainfall correction coefficient decreases as the rainfall increases.

[0096] The light intensity correction coefficient is a constant between 0 and 1, which can be obtained through pre-calibration. The light intensity correction coefficient decreases as the light intensity decreases. When the light intensity is greater than or equal to the preset light intensity threshold, it is regarded that the vehicle is in the daytime when the front road image is collected, and the light intensity correction coefficient is 1. When the light intensity is less than the light intensity threshold, it is regarded that the vehicle is in the night when the front road image is collected.

[0097] Step 405: Correct the original confidence according to the influence degree to obtain the first confidence; wherein, the influence degree is negatively correlated with the first confidence.

[0098] Among them, step 405 is substantially the same as the foregoing step 304 and will not be elaborated here.

[0099] Step 406: Determine a second prediction result of the road type according to the driving state parameter.

[0100] Among them, step 406 is substantially the same as the foregoing step 103 and will not be elaborated here.

[0101] Step 407: Determine the target type of the road where the vehicle is currently located according to the first prediction result and the second prediction result.

[0102] Among them, step 407 is substantially the same as the foregoing step 104 and will not be elaborated here.

[0103] In some embodiments, considering that there are differences in the representation methods and principles of different modal data for the road type of the vehicle, in order to improve the recognition accuracy of the road type of the vehicle based on multi-modal data, as Figure 5 shown, the road type recognition process includes: Step 501: Obtain the current driving state parameter of the vehicle and the front road image of the vehicle at the target time; the target time is determined according to the vehicle speed and the acquisition position difference of the vehicle; the target time is determined according to the vehicle speed and the acquisition position difference of the vehicle; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameter.

[0104] Among them, step 501 is substantially the same as the foregoing step 101 and will not be elaborated here.

[0105] Step 502: Identify the front road image to obtain a first prediction result of the road type of the road where the vehicle is currently located; the first prediction result includes the first confidence of the vehicle being in multiple preset road types.

[0106] Among them, the content of the preset road types and the process of recognizing the front road image can refer to the aforementioned step 102, which will not be elaborated here. For each preset road type, the first confidence corresponding to the preset road type is used to represent the probability that the vehicle is currently in this road type.

[0107] Step 503: Determine a second prediction result of the road type according to the driving state parameter. The second prediction result includes a second confidence of the vehicle currently being in the multiple preset road types.

[0108] Among them, the content of the preset road types and the process of recognizing the driving state parameter can refer to the aforementioned step 102, which will not be elaborated here. For each preset road type, the second confidence corresponding to the preset road type is used to represent the probability that the vehicle is currently in this road type.

[0109] Step 504: Compare the first confidences corresponding to each of the preset road types, and determine the preset road type with the largest first confidence as the first candidate type.

[0110] Among them, in order to improve the recognition accuracy of the vehicle's road type obtained when detecting based on the principle that different types of roads have differences in visual features, for the visual recognition result of the front road image, the preset road type with the largest first confidence is determined as the first candidate type.

[0111] Step 505: Compare the second confidences corresponding to each of the preset road types, and determine the preset road type with the largest second confidence as the second candidate type.

[0112] Among them, in order to improve the recognition accuracy of the vehicle's road type obtained when detecting based on the principle that the driving states of the vehicle on different types of roads are different, for the visual recognition result of the front road image, the preset road type with the largest first confidence is determined as the first candidate type.

[0113] Step 506: Determine the target type of the vehicle's current road according to the first candidate type, the second candidate type, and a preset confidence threshold.

[0114] Among them, considering that the first prediction result and the second prediction result are based on different detection principles. The first prediction result is based on the fact that different road types have different visual appearances, such as on snow or mud, there will be snow or mud on the road, while the second prediction result is based on the fact that the vehicle will switch to different dynamic states when driving on different roads to respond to the terrain of the road and ensure driving safety and riding experience.

[0115] Therefore, in the embodiments of the present invention, in order to ensure the accuracy of road type recognition based on multimodal data, the first confidence corresponding to the first candidate type is compared with the second confidence corresponding to the second candidate type to obtain the candidate type with a greater corresponding confidence. Then, the confidence corresponding to this candidate type is compared with a preset confidence threshold. If the confidence corresponding to this candidate type is greater than or equal to the confidence threshold, the candidate type is determined as the target type of the road where the vehicle is currently located, thereby avoiding large errors in visual recognition and driving state recognition and further ensuring the accuracy of target type recognition. For example, the first candidate type can be snow, and its corresponding first confidence is 0.85. The second candidate type can be ice, and its corresponding second confidence is 0.75. The preset confidence threshold can be 0.80, then the target type is snow.

[0116] Please refer to Figure 6 , which is a schematic structural diagram of the road type recognition device 60 provided in the embodiments of the present application. As Figure 6 shown, the road type recognition device 60 includes: An acquisition module 601, configured to acquire the current driving state parameters of the vehicle and the front road image of the vehicle at a target time; the target time is determined according to the vehicle speed and the acquisition position difference of the vehicle; the target time is determined according to the vehicle speed and the acquisition position difference of the vehicle; the acquisition position difference represents the distance between the first acquisition position of the front road image and the second acquisition position of the driving state parameters; An identification module 602, configured to identify the front road image to obtain a first prediction result of the road type of the road where the vehicle is currently located; A first determination module 603, configured to determine a second prediction result of the road type according to the driving state parameters; A second determination module 604, configured to determine the target type of the road where the vehicle is currently located according to the first prediction result and the second prediction result.

[0117] Please refer to Figure 7 , which is a schematic hardware structure diagram of the vehicle 70 provided in the embodiments of the present application. As Figure 7 shown, the vehicle 70 may include a processor 701 and a memory 702. The memory 702 is used to store one or more computer programs 703. The one or more computer programs 703 are configured to be executed by the processor 701. The one or more computer programs 703 include instructions, and the above instructions can be used to implement the control method of the in-vehicle device executed in the vehicle 701.

[0118] It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the vehicle 70. In some other embodiments, the vehicle 70 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements.

[0119] The processor 701 may include one or more processing units. For example, the processor 701 may include an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0120] The processor 701 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 701 is a cache memory. This memory can save the instructions or data that the processor 701 has just used or recycled. If the processor 701 needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 701, and thus improves the efficiency of the system.

[0121] In some embodiments, the processor 701 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0122] In some embodiments, the processor 701 is used to execute acceleration schemes such as single instruction multiple data (SIMD) and very long instruction word (VLIW).

[0123] In some embodiments, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, internal memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0124] This embodiment also provides a computer-readable storage medium storing computer instructions that, when run on a processor, cause the processor to execute the above-related method steps to implement the control method of the in-vehicle device in the above embodiment.

[0125] Wherein, the vehicle, device, and computer-readable storage medium provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be elaborated here.

[0126] In practical applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0127] In several embodiments provided in the present application, the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are illustrative. For example, the division of the module or unit is a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical, or other form.

[0128] The unit described as a separate component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0131] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application.

Claims

1. A road type recognition method, characterized in that: The method comprises: Acquire the current driving state parameters of the vehicle and the image of the road ahead of the vehicle at a target time; the target time is determined according to the vehicle speed and the acquisition position difference; the acquisition position difference represents the distance between a first acquisition position of the image of the road ahead and a second acquisition position of the driving state parameters; Identify the front road image to obtain a first prediction result of the road type of the road currently located by the vehicle; Determining a second prediction result of the road type according to the driving state parameter; The target type of the current road of the vehicle is determined according to the first prediction result and the second prediction result.

2. The method according to claim 1, characterized in that The target time is one of a plurality of preset candidate times; the vehicle speed includes the speed of the vehicle at each of the candidate times; The plurality of preset candidate times are earlier than the current time; The process of determining the target time includes: For each of the candidate times, determining, according to the speed corresponding to each of the candidate times and the acquisition position difference, an actual arrival time of the vehicle at the road area represented by the front road image acquired at each of the candidate times; The actual arrival time corresponding to each of the candidate times is matched with the current time, and the candidate time corresponding to the actual arrival time matching the current time is used as the target time.

3. The method according to claim 1, characterized in that: The first prediction result includes a first confidence level that the vehicle is currently on a preset road type; and a process for determining the first prediction result includes: Recognize the front road image to obtain an original confidence that the vehicle is currently on the preset road type; Analyzing the driving environment of the vehicle to obtain the degree of influence of the driving environment on the recognition accuracy of the front road image; The original confidence is modified according to the influence degree to obtain the first confidence degree; wherein the influence degree is negatively correlated with the first confidence degree.

4. The method according to claim 3, characterized in that The process of determining the degree of impact includes: Analyzing the driving environment of the vehicle according to preset harsh environment characteristics to obtain the harshness of the driving environment; the harsh environment characteristics include environmental characteristics that affect the visibility of the driving environment; The impact degree is determined according to the severity degree; wherein the severity degree is positively correlated with the impact degree.

5. The method according to claim 1, characterized in that The first prediction result includes a first confidence level that the vehicle is currently on a plurality of preset road types; the second prediction result includes a second confidence level that the vehicle is currently on the plurality of preset road types; The process of determining the target type includes: Compare the first confidences corresponding to the preset road types, and determine the preset road type with the largest first confidence as the first candidate type; Compare the second confidences corresponding to the preset road types, and determine the preset road type with the largest second confidence as the second candidate type; The target type is determined according to the first candidate type, the second candidate type, and a preset confidence threshold.

6. The method according to any one of claims 1 to 5, characterized in that: The driving state parameters include chassis state parameters and / or power state parameters of the vehicle; the correlation between the chassis state parameters and / or power state parameters and the driver's control operation of the vehicle is less than a preset threshold.

7. The method according to claim 6, characterized in that The chassis state parameters and / or power state parameters include at least one of the vehicle's four-wheel speed signals, vehicle speed signals, braking torque, steering angle, steering angular velocity, longitudinal acceleration, lateral acceleration, yaw angular velocity, vehicle driving torque, and four-wheel suspension height.

8. A road type recognition device, characterized in that: The road type identification device comprises: an acquisition module, used to acquire the current driving state parameters of the vehicle and the image of the road ahead of the vehicle at a target time; the target time is determined according to the vehicle speed and the acquisition position difference; the target time is determined according to the vehicle speed and the acquisition position difference; the acquisition position difference represents the distance between a first acquisition position of the image of the road ahead and a second acquisition position of the driving state parameters; A recognition module, used to recognize the front road image and obtain a first prediction result of the road type of the road currently located by the vehicle; A first determination module, used for determining a second prediction result of the road type according to the driving state parameter; A second determination module is configured to determine a target type of the current road of the vehicle based on the first prediction result and the second prediction result.

9. A vehicle, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory so that the vehicle executes the road type recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a processor, the processor is caused to execute the road type identification method according to any one of claims 1 to 7.

Images