Unmanned vehicle automatic control system and method

Abstract

According to the unmanned vehicle automatic control system and method provided by the invention, the vehicle cabin image in the vehicle is acquired and analyzed, and the internal personnel state information and the internal environment state information of the vehicle are obtained, so that the sensing equipment working mode and the vehicle cabin environment state of the vehicle are adjusted; it is ensured that the sensing equipment detects the external environment in a matched state and improves the driving environment in a vehicle cabin; detecting the driving environment of the vehicle in the unmanned driving state to obtain visual driving environment data and non-visual driving environment data, so as to determine corrected driving environment obstacle information and provide guidance for the unmanned driving state change of the vehicle; and on the basis of the corrected driving environment obstacle information, determining a driving route allowed to be changed corresponding to the driving change of the vehicle, adjusting the driving state of the vehicle in combination with the real-time driving state of the vehicle, and carrying out full-range detection on the driving environment of the vehicle by utilizing mutual linkage of different sensing devices. And the reliability and the safety of automatic driving of the vehicle are improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving, and in particular to an automatic control system and method for an unmanned vehicle. Background Art

[0002] Unmanned driving has become an important direction for the development of intelligent driving. Unmanned driving mainly relies on a large number of sensor devices installed in different positions of the vehicle to detect the external environment of the vehicle and locate obstacles that the vehicle may encounter during driving, so that the vehicle can adjust the driving direction and speed in time. The sensor devices installed in existing unmanned vehicles work independently of each other, and they cannot perform joint detection on the same external environment area, which reduces the accuracy of obstacle detection in the vehicle's driving environment, and cannot provide a full-range navigation map for the vehicle's driving, so that the vehicle cannot change its driving state safely and reliably. Summary of the invention

[0003] The purpose of the present invention is to provide an automatic control system and method for an unmanned vehicle, which acquires and analyzes the cabin image inside the vehicle, obtains the internal personnel status information and internal environment status information of the vehicle, and adjusts the working mode of the vehicle's sensor equipment and the cabin environment status to ensure that the sensor equipment performs external environment detection and improves the driving environment inside the cabin in a matching state; then detects the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data, thereby determining the corrected driving environment obstacle information to provide accurate guidance for the change of the vehicle's unmanned driving state; and based on the corrected driving environment obstacle information, determines the allowed change of driving route corresponding to the vehicle's driving change, and adjusts the vehicle's driving state in combination with the vehicle's real-time driving state, and uses different sensor equipment to work together to perform full-range detection of the vehicle's driving environment, thereby improving the reliability and safety of the vehicle's automatic driving.

[0004] The present invention is achieved through the following technical solutions:

[0005] An automatic control system for an unmanned vehicle, comprising:

[0006] A cabin image acquisition and analysis module, used to acquire cabin images inside the vehicle, analyze the cabin images, and obtain internal personnel status information and internal environment status information of the vehicle;

[0007] A sensor device adjustment module, used to adjust the working mode of the sensor device of the vehicle based on the internal personnel status information;

[0008] A cabin environment adjustment module, configured to adjust the cabin environment state of the vehicle based on the internal environment state information;

[0009] A driving environment detection control module is used to detect the driving environment of the vehicle in an unmanned driving state and obtain visual driving environment data and non-visual driving environment data;

[0010] A driving environment obstacle information determination module, used to determine the driving environment obstacle information of the vehicle based on the visual driving environment data; and then modify the driving environment obstacle information based on the non-visual driving environment data;

[0011] A driving route change determination module, used to determine the allowed driving route change corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information;

[0012] The driving state adjustment module is used to adjust the driving state of the vehicle based on the allowed change of the driving route and the real-time driving state of the vehicle.

[0013] Optionally, the cabin image acquisition and analysis module is used to acquire a cabin image inside the vehicle, analyze the cabin image, and obtain internal personnel status information and internal environment status information of the vehicle, including:

[0014] Scan and photograph the interior of the vehicle to obtain a cabin scan image; perform screen capture processing on the cabin scan image to obtain a dynamic image of a person in the driver's seat of the vehicle and a dynamic image of the front windshield of the vehicle;

[0015] The sensor device adjustment module is used to adjust the working mode of the sensor device of the vehicle based on the internal personnel status information, including:

[0016] Analyze the dynamic image of the person to obtain the motion state information of the person in the driver's seat; wherein the motion state information of the person includes the motion state information of the person's hands and head; based on the motion state information of the person, determine whether the person inside the vehicle is in a state where the vehicle can be controlled at any time; and based on the determination result of whether the person inside the vehicle is in a state where the vehicle can be controlled at any time, adjust the detection frequency and / or detection sensitivity of the sensor device of the vehicle;

[0017] The cabin environment adjustment module is used to adjust the cabin environment state of the vehicle based on the internal environment state information, including:

[0018] The dynamic image of the front windshield is analyzed to obtain the field of view clarity information of the front windshield of the vehicle; based on the field of view clarity information, the cabin temperature and / or cabin air flow circulation state of the vehicle are adjusted.

[0019] Optionally, the driving environment detection control module is used to detect the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data, including:

[0020] Based on the working mode of the sensor equipment of the vehicle, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range;

[0021] The driving environment obstacle information determination module is used to determine the driving environment obstacle information of the vehicle based on the visual driving environment data; and then modify the driving environment obstacle information based on the non-visual driving environment data, including:

[0022] Performing image recognition on the visual driving environment data to obtain information on the existence status of driving environment obstacles within the same detection range; wherein the size and location information of the driving environment obstacles;

[0023] The radar ranging data of the same detection range is extracted from the non-visual driving environment data, and based on the radar ranging data, the obstacle existence status information of the driving environment is corrected to obtain the closest distance and relative direction between the obstacle existing in the same detection range and the vehicle.

[0024] Optionally, the driving route change determination module is used to determine the allowed driving route change corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information, including:

[0025] Integrate the closest distances and relative positions of the corrected obstacles around the vehicle and the vehicle to obtain a distribution map of obstacles around the vehicle; based on the distribution map of obstacles around the vehicle, determine a path trajectory corresponding to the allowed change of driving route when the vehicle is avoiding the obstacle ahead;

[0026] The driving state adjustment module is used to adjust the driving state of the vehicle based on the allowed driving route change and the real-time driving state of the vehicle, including:

[0027] Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted so that the vehicle switches to the allowed driving route change.

[0028] An automatic control method for an unmanned vehicle, comprising:

[0029] Acquire a cabin image of the interior of the vehicle, analyze the cabin image, and obtain internal personnel status information and internal environment status information of the vehicle; adjust the working mode of the sensor device of the vehicle based on the internal personnel status information; and adjust the cabin environment status of the vehicle based on the internal environment status information;

[0030] Detecting the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data; determining driving environment obstacle information of the vehicle based on the visual driving environment data; and then correcting the driving environment obstacle information based on the non-visual driving environment data;

[0031] Based on the corrected driving environment obstacle information, the allowed change driving route corresponding to the driving change of the vehicle is determined; based on the allowed change driving route and the real-time driving state of the vehicle, the driving state of the vehicle is adjusted.

[0032] Optionally, acquiring a cabin image of a vehicle, analyzing the cabin image to obtain internal personnel status information and internal environment status information of the vehicle; adjusting a working mode of a sensor device of the vehicle based on the internal personnel status information; and adjusting a cabin environment status of the vehicle based on the internal environment status information, includes:

[0033] Scan and photograph the interior of the vehicle to obtain a cabin scan image; perform screen capture processing on the cabin scan image to obtain a dynamic image of a person in the driver's seat of the vehicle and a dynamic image of the front windshield of the vehicle;

[0034] Analyze the dynamic image of the person to obtain the motion state information of the person in the driver's seat; wherein the motion state information of the person includes the motion state information of the person's hands and head; based on the motion state information of the person, determine whether the person inside the vehicle is in a state where the vehicle can be controlled at any time; and based on the determination result of whether the person inside the vehicle is in a state where the vehicle can be controlled at any time, adjust the detection frequency and / or detection sensitivity of the sensor device of the vehicle;

[0035] The dynamic image of the front windshield is analyzed to obtain the field of view clarity information of the front windshield of the vehicle; based on the field of view clarity information, the cabin temperature and / or cabin air flow circulation state of the vehicle are adjusted.

[0036] Optionally, analyzing the dynamic image of the front windshield to obtain field of view clarity information of the front windshield of the vehicle; and adjusting the cabin temperature of the vehicle based on the field of view clarity information includes:

[0037] Step S1, using the following formula (1), according to the dynamic picture of the front windshield, obtain the field of view clarity of the front windshield of the vehicle,

[0038]

[0039] In the above formula (1), μ represents the field of view clarity of the front windshield of the vehicle; [R(a), G(a), B(a)] represents the RGB value of the ath pixel in the image matrix of the dynamic picture of the front windshield of the vehicle; [R0, G0, B0] represents the standard RGB value of the fogging point after the front windshield of the vehicle is fogged; n represents the total number of pixels in the image matrix of the dynamic picture of the front windshield of the vehicle; F() and F[] both represent judgment functions. If the formula in the brackets is established, the function value of the judgment function is 1, and if the formula in the brackets is not established, the function value of the judgment function is 0;

[0040] Step S2, using the following formula (2), based on the field of view clarity information and the outdoor temperature, determining whether the cabin temperature of the vehicle needs to be raised or lowered,

[0041]

[0042] In the above formula (2), W represents the control value of whether the cabin temperature of the vehicle is increased or decreased; Q represents the outdoor temperature;

[0043] If W=1, it is determined that the cabin temperature of the vehicle needs to be cooled;

[0044] If W=0, it is determined that the cabin temperature of the vehicle needs to be increased;

[0045] Step S3, when the field of view clarity is within the preset clarity range, the final adjustment target temperature of the cabin is determined according to the outdoor temperature and the number of people in the vehicle using the following formula (3):

[0046]

[0047] In the above formula (3), q represents the final target temperature of the cabin when the visual clarity is within the preset clarity range; M represents the maximum number of people that can be accommodated in the vehicle; q min Indicates the lowest temperature value that the air conditioner in the vehicle is allowed to adjust to; q max Indicates the maximum temperature value to which the air conditioner in the vehicle is allowed to be adjusted.

[0048] Optionally, detecting the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data; determining driving environment obstacle information of the vehicle based on the visual driving environment data; and then correcting the driving environment obstacle information based on the non-visual driving environment data, including:

[0049] Based on the working mode of the sensor equipment of the vehicle, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range;

[0050] Performing image recognition on the visual driving environment data to obtain information on the existence status of driving environment obstacles within the same detection range; wherein the size and location information of the driving environment obstacles;

[0051] The radar ranging data of the same detection range is extracted from the non-visual driving environment data, and based on the radar ranging data, the obstacle existence status information of the driving environment is corrected to obtain the closest distance and relative direction between the obstacle existing in the same detection range and the vehicle.

[0052] Optionally, determining an allowable changeable driving route corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information; and adjusting the driving state of the vehicle based on the allowable changeable driving route and the real-time driving state of the vehicle, comprises:

[0053] Integrate the closest distances and relative positions of the corrected obstacles around the vehicle and the vehicle to obtain a distribution map of obstacles around the vehicle; based on the distribution map of obstacles around the vehicle, determine a path trajectory corresponding to the allowed change of driving route when the vehicle is avoiding the obstacle ahead;

[0054] Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted so that the vehicle switches to the allowed driving route change.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The unmanned vehicle automatic control system and method provided in the present application acquire and analyze the cabin image inside the vehicle, obtain the internal personnel status information and internal environment status information of the vehicle, thereby adjusting the working mode of the vehicle's sensor equipment and the cabin environment status, and ensuring that the sensor equipment performs external environment detection and improves the driving environment inside the cabin in a matching state; then the driving environment of the vehicle in the unmanned driving state is detected to obtain visual driving environment data and non-visual driving environment data, thereby determining the corrected driving environment obstacle information, and providing accurate guidance for the change of the vehicle's unmanned driving state; based on the corrected driving environment obstacle information, the allowed driving route change corresponding to the vehicle's driving change is determined, and the vehicle's driving state is adjusted in combination with the vehicle's real-time driving state, and different sensor equipment are used to work together to perform a full-range detection of the vehicle's driving environment, thereby improving the reliability and safety of the vehicle's automatic driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0058] Figure 1 A schematic structural diagram of an unmanned vehicle automatic control system provided by the present invention.

[0059] Figure 2 A schematic flow chart of an automatic control method for an unmanned vehicle provided by the present invention. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0061] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0062] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0063] See also Figure 1 As shown, an embodiment of the present application provides an unmanned vehicle automatic control system. The unmanned vehicle automatic control system includes:

[0064] The cabin image acquisition and analysis module is used to acquire the cabin image inside the vehicle, analyze the cabin image, and obtain the internal personnel status information and internal environment status information of the vehicle;

[0065] A sensor device adjustment module, used to adjust the working mode of the sensor device of the vehicle based on the internal personnel status information;

[0066] A cabin environment adjustment module, used to adjust the cabin environment state of the vehicle based on the internal environment state information;

[0067] A driving environment detection control module is used to detect the driving environment of the vehicle in an unmanned driving state and obtain visual driving environment data and non-visual driving environment data;

[0068] A driving environment obstacle information determination module, used to determine the driving environment obstacle information of the vehicle based on the visual driving environment data; and then modify the driving environment obstacle information based on the non-visual driving environment data;

[0069] A driving route change determination module is used to determine the allowed driving route change corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information;

[0070] The driving state adjustment module is used to adjust the driving state of the vehicle based on the allowed change of the driving route and the real-time driving state of the vehicle.

[0071] The beneficial effects of the above embodiments are as follows: the unmanned vehicle automatic control system acquires and analyzes the cabin image inside the vehicle, obtains the internal personnel status information and internal environment status information of the vehicle, thereby adjusting the working mode of the vehicle's sensor equipment and the cabin environment status, ensuring that the sensor equipment performs external environment detection and improves the driving environment inside the cabin in a matching state; then the driving environment of the vehicle in the unmanned driving state is detected to obtain visual driving environment data and non-visual driving environment data, thereby determining the corrected driving environment obstacle information, and providing accurate guidance for the change of the vehicle's unmanned driving state; based on the corrected driving environment obstacle information, the allowed driving route change corresponding to the vehicle's driving change is determined, and the vehicle's driving state is adjusted in combination with the vehicle's real-time driving state, and different sensor equipment are used to work together to perform a full-range detection of the vehicle's driving environment, thereby improving the reliability and safety of the vehicle's automatic driving.

[0072] In another embodiment, the cabin image acquisition and analysis module is used to acquire the cabin image inside the vehicle, analyze the cabin image, and obtain the internal personnel status information and internal environment status information of the vehicle, including:

[0073] Scan and photograph the interior of the vehicle to obtain a cabin scan image; perform screen capture processing on the cabin scan image to obtain a dynamic image of a person in the driver's seat of the vehicle and a dynamic image of the front windshield of the vehicle;

[0074] The sensor device adjustment module is used to adjust the working mode of the sensor device of the vehicle based on the internal personnel status information, including:

[0075] Analyze the dynamic image of the person to obtain the motion state information of the person in the driver's seat; wherein the motion state information of the person includes the motion state information of the person's hands and head; based on the motion state information of the person, determine whether the person inside the vehicle is in a state where the vehicle can be controlled at any time; and based on the determination result of whether the person inside the vehicle is in a state where the vehicle can be controlled at any time, adjust the detection frequency and / or detection sensitivity of the sensor device of the vehicle;

[0076] The cabin environment adjustment module is used to adjust the cabin environment state of the vehicle based on the internal environment state information, including:

[0077] The dynamic image of the front windshield is analyzed to obtain the field of view clarity information of the front windshield of the vehicle; based on the field of view clarity information, the cabin temperature and / or cabin air flow circulation state of the vehicle are adjusted.

[0078] The beneficial effect of the above embodiment is that a panoramic scan is taken of the interior of the vehicle cabin to obtain a cabin scan image, and the cabin scan image is processed to obtain a corresponding dynamic image of the area where the driver's seat and the front windshield are located inside the vehicle, thereby dynamically visually identifying the status of the personnel inside the vehicle and the status of the front windshield. The dynamic image of the personnel is analyzed to obtain the action status information of the personnel on the driver's seat, so that the driving action status of the personnel on the driver's seat can be accurately identified, and it can be accurately judged whether the personnel inside the vehicle are in a state where they can control the vehicle at any time. Specifically, when the personnel inside the vehicle are not in a state where they can control the vehicle at any time (for example, the personnel's hands are not holding the steering wheel or the personnel's line of sight is not turned to the front of the vehicle), the detection frequency and / or detection sensitivity of all sensor devices of the vehicle are increased, so that the vehicle can be guaranteed to detect the external driving environment with higher detection performance in the unmanned driving state. The dynamic image of the windshield is also analyzed to obtain the field of view clarity information of the vehicle's windshield (i.e., the haze value of the windshield; the greater the haze of the current windshield, the smaller the corresponding field of view clarity). Based on the field of view clarity information, the vehicle's cabin temperature is increased or decreased and / or the cabin air flow circulation speed is increased. This can effectively remove the fog on the surface of the windshield, thereby improving the field of view clarity of the windshield.

[0079] In another embodiment, the driving environment detection control module is used to detect the driving environment of the vehicle in an unmanned driving state, and obtain visual driving environment data and non-visual driving environment data, including:

[0080] Based on the working mode of the sensor equipment of the vehicle, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range;

[0081] The driving environment obstacle information determination module is used to determine the driving environment obstacle information of the vehicle based on the visual driving environment data; and then to correct the driving environment obstacle information based on the non-visual driving environment data, including:

[0082] Performing image recognition on the visual driving environment data to obtain information on the existence status of driving environment obstacles within the same detection range; wherein the size and location information of the driving environment obstacles;

[0083] The radar ranging data for the same detection range is extracted from the non-visual driving environment data, and based on the radar ranging data, the obstacle existence status information of the driving environment is corrected to obtain the closest distance and relative direction between the obstacle existing in the same detection range and the vehicle.

[0084] The beneficial effect of the above embodiment is that, based on the working mode of the vehicle's sensor equipment, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, wherein the visual sensor equipment may be but not limited to a wide-angle camera, and the non-visual sensor equipment may be but not limited to a scanning radar, so that the external driving environment of the vehicle can be visually and non-visually detected, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range, ensuring the consistency of the content of the visual driving environment data and the non-visual driving environment data. The visual driving environment data is also subjected to image recognition to obtain the existence status information of driving environment obstacles within the same detection range, the size and existence position information of the driving environment obstacles, and then the radar ranging data of the same detection range is extracted from the non-visual driving environment data. The existence status information of the driving environment obstacles is corrected using the radar ranging data, so as to obtain the closest distance and relative direction between the obstacles existing in the same detection range and the vehicle, thereby realizing comprehensive and accurate positioning of the relative position relationship between the vehicle and the obstacles.

[0085] In another embodiment, the driving route change determination module is used to determine the allowed driving route change corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information, including:

[0086] Integrate the closest distances and relative positions of the corrected obstacles around the vehicle and the vehicle to obtain a distribution map of obstacles around the vehicle; based on the distribution map of obstacles around the vehicle, determine a path trajectory corresponding to the allowed change of driving route when the vehicle is making driving changes in the process of avoiding the obstacle ahead;

[0087] The driving state adjustment module is used to adjust the driving state of the vehicle based on the allowed driving route change and the real-time driving state of the vehicle, including:

[0088] Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted so that the vehicle switches to the allowed driving route change.

[0089] The beneficial effect of the above embodiment is that the closest distance and relative position of the corrected obstacles around the vehicle are integrated to obtain a map of obstacle distribution around the vehicle, so that the distribution of obstacles around the vehicle can be accurately identified. Based on the map of obstacle distribution around the vehicle, the path trajectory of the allowed driving route change corresponding to the driving change made by the vehicle in the process of avoiding the obstacle ahead is determined, so that a reliable guidance basis can be provided for the vehicle to change the driving route in time to avoid obstacles in the case of unmanned driving. Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted, so that the vehicle switches to the allowed driving route change, thereby improving the safety and reliability of the unmanned driving of the vehicle.

[0090] See also Figure 2 As shown, an embodiment of the present application provides an automatic control method for an unmanned vehicle. The automatic control method for an unmanned vehicle includes:

[0091] Acquire a cabin image of the vehicle, analyze the cabin image, and obtain internal personnel status information and internal environment status information of the vehicle; adjust the working mode of the sensor equipment of the vehicle based on the internal personnel status information; and adjust the cabin environment status of the vehicle based on the internal environment status information;

[0092] Detecting the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data; determining driving environment obstacle information of the vehicle based on the visual driving environment data; and then correcting the driving environment obstacle information based on the non-visual driving environment data;

[0093] Based on the corrected driving environment obstacle information, the allowed change driving route corresponding to the driving change of the vehicle is determined; based on the allowed change driving route and the real-time driving state of the vehicle, the driving state of the vehicle is adjusted.

[0094] The beneficial effects of the above embodiments are as follows: the unmanned vehicle automatic control method acquires and analyzes the cabin image inside the vehicle, obtains the internal personnel status information and internal environment status information of the vehicle, thereby adjusting the working mode of the vehicle's sensor equipment and the cabin environment status, ensuring that the sensor equipment performs external environment detection and improves the driving environment inside the cabin in a matching state; then the driving environment of the vehicle in the unmanned driving state is detected to obtain visual driving environment data and non-visual driving environment data, thereby determining the corrected driving environment obstacle information, and providing accurate guidance for the change of the vehicle's unmanned driving state; based on the corrected driving environment obstacle information, the allowed change of driving route corresponding to the vehicle's driving change is determined, and the vehicle's driving state is adjusted in combination with the vehicle's real-time driving state, and different sensor equipment are used to work together to perform a full-range detection of the vehicle's driving environment, thereby improving the reliability and safety of the vehicle's automatic driving.

[0095] In another embodiment, obtaining a cabin image of a vehicle, analyzing the cabin image to obtain internal personnel status information and internal environment status information of the vehicle; adjusting a working mode of a sensor device of the vehicle based on the internal personnel status information; and adjusting a cabin environment status of the vehicle based on the internal environment status information include:

[0096] Scan and photograph the interior of the vehicle to obtain a cabin scan image; perform screen capture processing on the cabin scan image to obtain a dynamic image of a person in the driver's seat of the vehicle and a dynamic image of the front windshield of the vehicle;

[0097] Analyze the dynamic image of the person to obtain the motion state information of the person in the driver's seat; wherein the motion state information of the person includes the motion state information of the person's hands and head; based on the motion state information of the person, determine whether the person inside the vehicle is in a state where the vehicle can be controlled at any time; and based on the determination result of whether the person inside the vehicle is in a state where the vehicle can be controlled at any time, adjust the detection frequency and / or detection sensitivity of the sensor device of the vehicle;

[0098] The dynamic image of the front windshield is analyzed to obtain the field of view clarity information of the front windshield of the vehicle; based on the field of view clarity information, the cabin temperature and / or cabin air flow circulation state of the vehicle are adjusted.

[0099] The beneficial effect of the above embodiment is that a panoramic scan is taken of the interior of the vehicle cabin to obtain a cabin scan image, and the cabin scan image is processed to obtain a corresponding dynamic image of the area where the driver's seat and the front windshield are located inside the vehicle, thereby dynamically visually identifying the status of the personnel inside the vehicle and the status of the front windshield. The dynamic image of the personnel is analyzed to obtain the action status information of the personnel on the driver's seat, so that the driving action status of the personnel on the driver's seat can be accurately identified, and it can be accurately judged whether the personnel inside the vehicle are in a state where they can control the vehicle at any time. Specifically, when the personnel inside the vehicle are not in a state where they can control the vehicle at any time (for example, the personnel's hands are not holding the steering wheel or the personnel's line of sight is not turned to the front of the vehicle), the detection frequency and / or detection sensitivity of all sensor devices of the vehicle are increased, so that the vehicle can be guaranteed to detect the external driving environment with higher detection performance in the unmanned driving state. The dynamic image of the windshield is also analyzed to obtain the field of view clarity information of the vehicle's windshield (i.e., the haze value of the windshield; the greater the haze of the current windshield, the smaller the corresponding field of view clarity). Based on the field of view clarity information, the vehicle's cabin temperature is increased or decreased and / or the cabin air flow circulation speed is increased. This can effectively remove the fog on the surface of the windshield, thereby improving the field of view clarity of the windshield.

[0100] In another embodiment, analyzing the dynamic image of the front windshield to obtain the field of view clarity information of the front windshield of the vehicle; and adjusting the cabin temperature of the vehicle based on the field of view clarity information, including:

[0101] Step S1, using the following formula (1), according to the dynamic picture of the front windshield, obtain the field of view clarity of the front windshield of the vehicle,

[0102]

[0103] In the above formula (1), μ represents the field of view clarity of the front windshield of the vehicle; [R(a), G(a), B(a)] represents the RGB value of the ath pixel in the image matrix of the dynamic picture of the front windshield of the vehicle; [R0, G0, B0] represents the standard RGB value of the fogging point after the front windshield of the vehicle is fogged; n represents the total number of pixels in the image matrix of the dynamic picture of the front windshield of the vehicle; F() and F[] both represent judgment functions. If the formula in the brackets is established, the function value of the judgment function is 1, and if the formula in the brackets is not established, the function value of the judgment function is 0;

[0104] Step S2, using the following formula (2), based on the field of view clarity information and the outdoor temperature, determine whether the cabin temperature of the vehicle needs to be increased or decreased.

[0105]

[0106] In the above formula (2), W represents the control value of whether the cabin temperature of the vehicle is increased or decreased; Q represents the outdoor temperature;

[0107] If W=1, it is determined that the cabin temperature of the vehicle needs to be cooled;

[0108] If W=0, it is determined that the cabin temperature of the vehicle needs to be raised;

[0109] Step S3: When the field of view clarity is within the preset clarity range, the final adjustment target temperature of the cabin is determined according to the outdoor temperature and the number of people in the vehicle using the following formula (3):

[0110]

[0111] In the above formula (3), q represents the final target temperature of the cabin when the visual clarity is within the preset clarity range; M represents the maximum number of people that can be accommodated in the vehicle; q min Indicates the lowest temperature value that the air conditioner in the vehicle is allowed to adjust to; q max Indicates the maximum temperature value that the air conditioner in the vehicle is allowed to be adjusted to.

[0112] The beneficial effect of the above embodiment is that the above formula (1) is used to obtain the field of view clarity of the front windshield of the vehicle according to the dynamic picture of the front windshield, thereby intelligently quantifying the field of view clarity of the front windshield, which reflects the intelligence of the system; then the above formula (2) is used to determine whether the cabin temperature of the vehicle needs to be heated up or cooled down based on the field of view clarity information and the outdoor temperature. Since cooling can easily eliminate glass fogging, when the field of view clarity is small, the field of view can be clarified in time by cooling down to avoid accidents. When the field of view clarity is large, the outdoor temperature is used to determine whether to raise or lower the temperature to enhance the comfort of the occupants; finally, the above formula (3) is used to determine the final target temperature of the cabin according to the outdoor temperature and the number of occupants in the vehicle, thereby preventing the temperature inside the vehicle from being high when the outdoor temperature is high and low when the outdoor temperature is low, causing discomfort to the occupants.

[0113] In another embodiment, the driving environment of the vehicle in an unmanned driving state is detected to obtain visual driving environment data and non-visual driving environment data; based on the visual driving environment data, driving environment obstacle information of the vehicle is determined; and based on the non-visual driving environment data, the driving environment obstacle information is corrected, including:

[0114] Based on the working mode of the sensor equipment of the vehicle, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range;

[0115] Performing image recognition on the visual driving environment data to obtain information on the existence status of driving environment obstacles within the same detection range; wherein the size and location information of the driving environment obstacles;

[0116] The radar ranging data for the same detection range is extracted from the non-visual driving environment data, and based on the radar ranging data, the obstacle existence status information of the driving environment is corrected to obtain the closest distance and relative direction between the obstacle existing in the same detection range and the vehicle.

[0117] The beneficial effect of the above embodiment is that, based on the working mode of the vehicle's sensor equipment, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, wherein the visual sensor equipment may be but not limited to a wide-angle camera, and the non-visual sensor equipment may be but not limited to a scanning radar, so that the external driving environment of the vehicle can be visually and non-visually detected, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range, ensuring the consistency of the content of the visual driving environment data and the non-visual driving environment data. The visual driving environment data is also subjected to image recognition to obtain the existence status information of driving environment obstacles within the same detection range, the size and existence position information of the driving environment obstacles, and then the radar ranging data of the same detection range is extracted from the non-visual driving environment data. The existence status information of the driving environment obstacles is corrected using the radar ranging data, so as to obtain the closest distance and relative direction between the obstacles existing in the same detection range and the vehicle, thereby realizing comprehensive and accurate positioning of the relative position relationship between the vehicle and the obstacles.

[0118] In another embodiment, determining an allowable change driving route corresponding to the vehicle's driving change based on the corrected driving environment obstacle information; and adjusting the driving state of the vehicle based on the allowable change driving route and the real-time driving state of the vehicle, including:

[0119] Integrate the closest distances and relative positions of the corrected obstacles around the vehicle and the vehicle to obtain a distribution map of obstacles around the vehicle; based on the distribution map of obstacles around the vehicle, determine a path trajectory corresponding to the allowed change of driving route when the vehicle is making driving changes in the process of avoiding the obstacle ahead;

[0120] Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted so that the vehicle switches to the allowed driving route change.

[0121] The beneficial effect of the above embodiment is that the closest distance and relative position of the corrected obstacles around the vehicle are integrated to obtain a map of obstacle distribution around the vehicle, so that the distribution of obstacles around the vehicle can be accurately identified. Based on the map of obstacle distribution around the vehicle, the path trajectory of the allowed driving route change corresponding to the driving change made by the vehicle in the process of avoiding the obstacle ahead is determined, so that a reliable guidance basis can be provided for the vehicle to change the driving route in time to avoid obstacles in the case of unmanned driving. Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted, so that the vehicle switches to the allowed driving route change, thereby improving the safety and reliability of the unmanned driving of the vehicle.

[0122] In general, the unmanned vehicle automatic control system and method acquire and analyze the cabin image inside the vehicle, obtain the internal personnel status information and internal environment status information of the vehicle, and adjust the working mode of the vehicle's sensor equipment and the cabin environment status to ensure that the sensor equipment performs external environment detection and improves the driving environment inside the cabin in a matching state; then detect the driving environment of the vehicle in the unmanned driving state to obtain visual driving environment data and non-visual driving environment data, thereby determining the corrected driving environment obstacle information, and providing accurate guidance for the change of the vehicle's unmanned driving state; based on the corrected driving environment obstacle information, determine the allowed change of driving route corresponding to the vehicle's driving change, and adjust the vehicle's driving state in combination with the vehicle's real-time driving state, and use different sensor equipment to work together to perform full-range detection of the vehicle's driving environment, thereby improving the reliability and safety of the vehicle's automatic driving.

[0123] The above is only a specific implementation of the present invention, and any other improvements made based on the concept of the present invention are considered to be within the protection scope of the present invention.

Claims

1. An automatic control system for an unmanned vehicle, characterized in that: include: A cabin image acquisition and analysis module, used to acquire cabin images inside the vehicle, analyze the cabin images, and obtain internal personnel status information and internal environment status information of the vehicle; a sensor device adjustment module, used to adjust the working mode of the sensor device of the vehicle based on the internal personnel status information; A cabin environment adjustment module, configured to adjust the cabin environment state of the vehicle based on the internal environment state information; A driving environment detection control module is used to detect the driving environment of the vehicle in an unmanned driving state and obtain visual driving environment data and non-visual driving environment data; A driving environment obstacle information determination module, used to determine the driving environment obstacle information of the vehicle based on the visual driving environment data; Then, based on the non-visual driving environment data, the driving environment obstacle information is corrected; A driving route change determination module, used to determine the allowed driving route change corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information; The driving state adjustment module is used to adjust the driving state of the vehicle based on the allowed change of the driving route and the real-time driving state of the vehicle.

2. The unmanned vehicle automatic control system according to claim 1, characterized in that: The cabin image acquisition and analysis module is used to acquire the cabin image inside the vehicle, analyze the cabin image, and obtain the internal personnel status information and internal environment status information of the vehicle, including: Scan and photograph the interior of the vehicle to obtain a cabin scan image; perform screen capture processing on the cabin scan image to obtain a dynamic image of a person in the driver's seat of the vehicle and a dynamic image of the front windshield of the vehicle; The sensor device adjustment module is used to adjust the working mode of the sensor device of the vehicle based on the internal personnel status information, including: Analyze the dynamic image of the person to obtain the motion state information of the person in the driver's seat; wherein the motion state information of the person includes the motion state information of the person's hands and head; based on the motion state information of the person, determine whether the person inside the vehicle is in a state where the vehicle can be controlled at any time; and based on the determination result of whether the person inside the vehicle is in a state where the vehicle can be controlled at any time, adjust the detection frequency and / or detection sensitivity of the sensor device of the vehicle; The cabin environment adjustment module is used to adjust the cabin environment state of the vehicle based on the internal environment state information, including: The dynamic image of the front windshield is analyzed to obtain the field of view clarity information of the front windshield of the vehicle; based on the field of view clarity information, the cabin temperature and / or cabin air flow circulation state of the vehicle are adjusted.

3. The automatic control system for an unmanned vehicle as claimed in claim 2, characterized in that: The driving environment detection control module is used to detect the driving environment of the vehicle in an unmanned driving state, and obtain visual driving environment data and non-visual driving environment data, including: Based on the working mode of the sensor equipment of the vehicle, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range; The driving environment obstacle information determination module is used to determine the driving environment obstacle information of the vehicle based on the visual driving environment data; and then modify the driving environment obstacle information based on the non-visual driving environment data, including: Performing image recognition on the visual driving environment data to obtain information on the existence status of driving environment obstacles within the same detection range; wherein the size and location information of the driving environment obstacles; The radar ranging data of the same detection range is extracted from the non-visual driving environment data, and based on the radar ranging data, the obstacle existence status information of the driving environment is corrected to obtain the closest distance and relative direction between the obstacle existing in the same detection range and the vehicle.

4. The automatic control system for an unmanned vehicle as claimed in claim 3, characterized in that: The driving route change determination module is used to determine the allowed driving route change corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information, including: Integrate the closest distances and relative positions of the corrected obstacles around the vehicle and the vehicle to obtain a distribution map of obstacles around the vehicle; based on the distribution map of obstacles around the vehicle, determine a path trajectory corresponding to the allowed change of driving route when the vehicle is avoiding the obstacle ahead; The driving state adjustment module is used to adjust the driving state of the vehicle based on the allowed driving route change and the real-time driving state of the vehicle, including: Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted so that the vehicle switches to the allowed driving route change.

5. An automatic control method for an unmanned vehicle, characterized in that: include: Acquire a cabin image of the interior of the vehicle, analyze the cabin image, and obtain internal personnel status information and internal environment status information of the vehicle; adjust the working mode of the sensor device of the vehicle based on the internal personnel status information; and adjust the cabin environment status of the vehicle based on the internal environment status information; Detecting the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data; determining driving environment obstacle information of the vehicle based on the visual driving environment data; Then, based on the non-visual driving environment data, the driving environment obstacle information is corrected; Determining an allowed change of driving route corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information; The driving state of the vehicle is adjusted based on the allowed change of the driving route and the real-time driving state of the vehicle.

6. The automatic control method of an unmanned vehicle according to claim 5, characterized in that: Acquire a cabin image of the vehicle, analyze the cabin image, and obtain internal personnel status information and internal environment status information of the vehicle; and adjust a working mode of a sensor device of the vehicle based on the internal personnel status information; Adjusting the cabin environment state of the vehicle based on the internal environment state information includes: Scan and photograph the interior of the vehicle to obtain a cabin scan image; perform screen capture processing on the cabin scan image to obtain a dynamic image of a person in the driver's seat of the vehicle and a dynamic image of the front windshield of the vehicle; Analyze the dynamic image of the person to obtain the motion state information of the person in the driver's seat; wherein the motion state information of the person includes the motion state information of the person's hands and head; based on the motion state information of the person, determine whether the person inside the vehicle is in a state where the vehicle can be controlled at any time; and based on the determination result of whether the person inside the vehicle is in a state where the vehicle can be controlled at any time, adjust the detection frequency and / or detection sensitivity of the sensor device of the vehicle; The dynamic image of the front windshield is analyzed to obtain the field of view clarity information of the front windshield of the vehicle; based on the field of view clarity information, the cabin temperature and / or cabin air flow circulation state of the vehicle are adjusted.

7. The automatic control method of an unmanned vehicle according to claim 6, characterized in that: Analyzing the dynamic image of the front windshield to obtain the field of view clarity information of the front windshield of the vehicle; Adjusting the cabin temperature of the vehicle based on the field of view clarity information includes: Step S1, using the following formula (1), according to the dynamic picture of the front windshield, obtain the field of view clarity of the front windshield of the vehicle, In the above formula (1), μ represents the field of view clarity of the front windshield of the vehicle; [R(a), G(a), B(a)] represents the RGB value of the ath pixel in the image matrix of the dynamic picture of the front windshield of the vehicle; [R0, G0, B0] represents the standard RGB value of the fogging point after the front windshield of the vehicle is fogged; n represents the total number of pixels in the image matrix of the dynamic picture of the front windshield of the vehicle; F() and F[] both represent judgment functions. If the formula in the brackets is established, the function value of the judgment function is 1, and if the formula in the brackets is not established, the function value of the judgment function is 0; Step S2, using the following formula (2), based on the field of view clarity information and the outdoor temperature, determining whether the cabin temperature of the vehicle needs to be raised or lowered, In the above formula (2), W represents the control value of whether the cabin temperature of the vehicle is increased or decreased; Q represents the outdoor temperature; If W=1, it is determined that the cabin temperature of the vehicle needs to be cooled; If W=0, it is determined that the cabin temperature of the vehicle needs to be increased; Step S3, when the field of view clarity is within the preset clarity range, the final adjustment target temperature of the cabin is determined according to the outdoor temperature and the number of people in the vehicle using the following formula (3): In the above formula (3), q represents the final target temperature of the cabin when the visual clarity is within the preset clarity range; M represents the maximum number of people that can be accommodated in the vehicle; q min Indicates the lowest temperature value that the air conditioner in the vehicle is allowed to adjust to; q max Indicates the maximum temperature value to which the air conditioner in the vehicle is allowed to be adjusted.

8. The automatic control method of an unmanned vehicle according to claim 6, characterized in that: Detecting the driving environment of the vehicle in an unmanned driving state to obtain visual driving environment data and non-visual driving environment data; determining driving environment obstacle information of the vehicle based on the visual driving environment data; Based on the non-visual driving environment data, the driving environment obstacle information is corrected, including: Based on the working mode of the sensor equipment of the vehicle, the detection range and detection action of the visual sensor equipment and the non-visual sensor equipment located in the same area of ​​the vehicle are adjusted, so that the visual sensor equipment and the non-visual sensor equipment located in the same area can synchronously detect the same detection range, thereby obtaining visual driving environment data and non-visual driving environment data about the same detection range; Performing image recognition on the visual driving environment data to obtain information on the existence status of driving environment obstacles within the same detection range; wherein the size and location information of the driving environment obstacles; The radar ranging data of the same detection range is extracted from the non-visual driving environment data, and based on the radar ranging data, the obstacle existence status information of the driving environment is corrected to obtain the closest distance and relative direction between the obstacle existing in the same detection range and the vehicle.

9. The automatic control method of an unmanned vehicle according to claim 8, characterized in that: Determining an allowed change of driving route corresponding to the driving change of the vehicle based on the corrected driving environment obstacle information; Adjusting the driving state of the vehicle based on the allowed driving route change and the real-time driving state of the vehicle includes: Integrate the closest distances and relative positions of the corrected obstacles around the vehicle and the vehicle to obtain a distribution map of obstacles around the vehicle; based on the distribution map of obstacles around the vehicle, determine a path trajectory corresponding to the allowed change of driving route when the vehicle is avoiding the obstacle ahead; Based on the path trajectory of the allowed driving route change and the real-time driving speed of the vehicle, the driving direction and driving speed of the vehicle are adjusted so that the vehicle switches to the allowed driving route change.

Images