Human-machine collaborative control method, device and system based on multi-source risk

Abstract

The application provides a kind of man-machine collaborative control method, device and system based on multi-source risk, it is related to remote driving control technical field, obtains the first control information, delay information and reaction information sent by operator end, and vehicle state information and second control information sent by vehicle end;Determine the first participation coefficient and the first distribution weight based on delay information and reaction information, and determine the second participation coefficient and the second distribution weight based on vehicle state information;Determine the influence parameter based on the first participation coefficient, the first distribution weight, the second participation coefficient and the second distribution weight;Determine the target control information for controlling vehicle based on influence parameter, first control information and second control information.The application comprehensively considers the multi-source risk factors existing in remote driving, realizes the collaborative decision of multi-factor fusion, solves the defects of single perspective, technical idealization and incomplete analysis factors.

Description

Technical Field

[0001] This invention relates to the field of remote driving control technology, and in particular to a human-machine collaborative control method, device and system based on multi-source risks. Background Technology

[0002] Remote driving is a driving mode based on cloud computing and control. It uses high-speed networks such as 5G to upload real-time road condition data collected by vehicle sensors to a cloud server, and a remote operator makes decisions in the cockpit. Then, the control commands are sent to the vehicle actuators through the cloud, thereby realizing remote and precise control of the vehicle.

[0003] When dealing with multi-source risks in remote driving, such as remote operator distraction, network latency, and driving field risks, there are generally limitations such as a single perspective, idealized technology, and incomplete analysis of factors. Summary of the Invention

[0004] This invention provides a human-machine collaborative control method, device, and system based on multi-source risks, which addresses the limitations of existing technologies in handling multi-source risks such as remote operator distraction, network latency, and driving potential field risks in remote driving, which are generally limited by a single perspective, idealized technology, and incomplete analysis of factors.

[0005] This invention provides a human-machine collaborative control method based on multi-source risk, applied in the cloud, comprising: Acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal; Based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle state information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, the influencing parameters are determined; Based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined.

[0006] According to a human-machine collaborative control method based on multi-source risk provided by the present invention, the method comprises determining a first participation coefficient and a first allocation weight based on the delay information and the response information, and determining a second participation coefficient and a second allocation weight based on the vehicle state information, including: The delay information and the reaction information are respectively input into the first risk quantification model and the fuzzy logic model to obtain the first allocation weight output by the first risk quantification model and the first participation coefficient output by the fuzzy logic model. The vehicle status information is input into the second risk quantification model and the fuzzy logic model respectively to obtain the second allocation weight output by the second risk quantification model and the second participation coefficient output by the fuzzy logic model.

[0007] According to the present invention, a human-machine collaborative control method based on multi-source risk is provided, wherein the first risk quantification model is a neural network model or a linear model obtained by least squares fitting, the second risk quantification model is a neural network model or an exponential model obtained by least squares fitting, and the fuzzy logic model is obtained based on a neural network model, a Bayesian network model or a linear fuzzy control model.

[0008] According to the present invention, a human-machine collaborative control method based on multi-source risk is provided. The fuzzy logic model is used to fuzzify the delay information and the response information, and to map the fuzzified delay information and response information based on fuzzy set mapping rules to obtain the delay-response degree. The delay-response degree is then defuzzified to obtain the first participation coefficient. The method is also used to fuzzify the potential field risk index and the tracking error index in the vehicle state information, and to map the fuzzified potential field risk index and the tracking error index based on fuzzy set mapping rules to obtain the potential field risk degree. The potential field risk degree is then defuzzified to obtain the second participation coefficient.

[0009] According to the present invention, a human-machine collaborative control method based on multi-source risk is provided, wherein determining the influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight includes: Determine the first product between the first participation coefficient and the first allocation weight; Determine the second product between the second participation coefficient and the second allocation weight; The sum of the first product and the second product is used as the influence parameter; The step of determining the target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information includes: Using the influence parameters as weights, the first control information and the second control information are weighted and fused to obtain target control information for controlling the vehicle.

[0010] The present invention also provides a human-machine collaborative control device based on multi-source risk, comprising: The acquisition module is used to acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal. The first determining module is used to determine a first participation coefficient and a first allocation weight based on the delay information and the response information, and to determine a second participation coefficient and a second allocation weight based on the vehicle state information. The second determining module is used to determine the influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight. The third determining module is used to determine target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information.

[0011] The present invention also provides a human-machine collaborative control system based on multi-source risk, implemented based on any of the above-described human-machine collaborative control methods based on multi-source risk, the system including a cloud, an operator terminal, and a vehicle terminal; The cloud platform is used to acquire first control information, delay information, and response information sent by the operator terminal, as well as vehicle status information and second control information sent by the vehicle terminal; based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle status information, a second participation coefficient and a second allocation weight are determined; based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, an influence parameter is determined; based on the influence parameter, the first control information, and the second control information, target control information for controlling the vehicle is determined. The vehicle is equipped with a model prediction controller, which is used to acquire the vehicle state information and determine the second control information based on the vehicle state information to complete the forward roll optimization of the vehicle. The operator terminal is equipped with a driver model, which is used to determine the first control information based on the delay information and the reaction information.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human-machine collaborative control method based on multi-source risk as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human-machine collaborative control method based on multi-source risk as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the human-machine collaborative control method based on multi-source risk as described above.

[0015] This invention provides a human-machine collaborative control method, device, and system based on multi-source risk. It acquires first control information, delay information, and reaction information sent from the operator's end, as well as vehicle status information and second control information sent from the vehicle's end. Based on the delay information and the reaction information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle status information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, influence parameters are determined. Based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined. This invention utilizes the delay and reaction information from the operator's end and the vehicle status information from the vehicle's end to determine influence parameters, and then uses the influence parameters, the first control information, and the second control information to determine the target control information for controlling the vehicle. It comprehensively considers the multi-source risk factors existing in remote driving, achieving collaborative decision-making through multi-factor fusion, and solving the shortcomings of a single perspective, idealized technology, and incomplete analysis of factors. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the human-machine collaborative control method based on multi-source risk provided by the present invention.

[0018] Figure 2 This is a schematic diagram of the structure of the fuzzy logic model provided by the present invention.

[0019] Figure 3 This is one of the schematic diagrams of fuzzy set mapping rules provided by the present invention.

[0020] Figure 4 This is the second schematic diagram of the fuzzy set mapping rules provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the structure of the human-machine collaborative control device based on multi-source risk provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the human-machine collaborative control system based on multi-source risk provided by the present invention.

[0023] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] In research on remote driving, network latency has always been a key factor. However, studies on shared control based on idealized network latency often fail to reflect the significant dangers posed by random fluctuations in network latency and even communication interruptions in real-world environments. Furthermore, most solutions only study multi-source risk factors independently, failing to comprehensively consider control performance under the simultaneous influence of multiple factors. Specifically, existing solutions either focus solely on optimizing network latency or only analyze driving characteristics and driving potential field risks, failing to integrate multiple factors to address end-to-end latency challenges. Moreover, current technologies do not fully consider the real-time latency analysis capabilities and local autonomous decision-making capabilities inherent in intelligent vehicles, still treating the vehicle as a passive executor rather than an intelligent collaborative entity capable of actively compensating for latency. This makes it impossible to predict and correct operational lags caused by latency, resulting in low control real-time performance and reliability. Driving potential field risk refers to treating the environment around the vehicle (such as obstacles and road boundaries) as a field with potential energy, and assessing collision risk by calculating the forces acting on the vehicle within this field.

[0026] To address the aforementioned problems, this invention provides a human-machine collaborative control scheme based on multi-source risks, which will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating the human-machine collaborative control method based on multi-source risk provided by the present invention, as shown below. Figure 1 As shown, the present invention provides a human-machine collaborative control method based on multi-source risk, applied in the cloud, for collaborative control of vehicles, the method including steps S110-S140.

[0028] Step S110: Obtain the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal.

[0029] Optionally, the first control information is used to characterize the operator-initiated instructions for controlling the vehicle, including control instructions such as steering, forward, backward, acceleration, and deceleration, or to characterize the operator-initiated control instructions optimized based on delay information and / or operator reaction information.

[0030] Optionally, during remote driving, the remote operator makes decisions based on road condition data collected by vehicle sensors, and the delay information is used to characterize the network delay information during the transmission of road condition data.

[0031] Optionally, the reaction information is used to characterize the time interval between the cockpit displaying road condition data to the driver and receiving control commands from the driver. If the time interval exceeds a time threshold, it can be determined that the driver is distracted.

[0032] Optionally, vehicle status information includes road condition data collected by vehicle sensors, which includes vehicle status data, surrounding environment perception data, traffic situation and map data.

[0033] Vehicle status data includes basic operating data (GPS / BeiDou vehicle position, speed, heading angle, acceleration, yaw rate, steering angle, etc.), key component status (battery charge, tire pressure, braking status, accelerator pedal opening, gear position, light status, etc.), and driving operation data (feedback data after the vehicle executes control commands issued by the remote driver).

[0034] Surrounding environment perception data includes dynamic targets (type, location, speed, and trajectory of surrounding vehicles, pedestrians, and non-motorized vehicles), static environment (lane line position, traffic signs, traffic light status, roadside facilities, etc.), and natural conditions (weather conditions, road surface conditions, and lighting conditions).

[0035] Traffic situation and map data include traffic situation (road congestion level, traffic flow, average vehicle speed, traffic accident or construction information, etc.) and high-precision maps (lane-level precision road models, static information such as slope and curvature, as well as dynamically updated road network information).

[0036] Optionally, the present invention deploys a model predictive controller on the vehicle side. By deploying the model predictive controller, forward roll optimization of the vehicle can be performed, which can effectively handle vehicle dynamic constraints and improve tracking performance and robustness. The second control information is used to characterize the vehicle control commands generated by the vehicle side based on the vehicle state information.

[0037] Step S120: Based on the delay information and the response information, determine the first participation coefficient and the first allocation weight, and based on the vehicle state information, determine the second participation coefficient and the second allocation weight.

[0038] Optionally, the risk value at the driver's end is obtained based on the delay information and the response information, and the first allocation weight is obtained based on the risk value at the driver's end. The first allocation weight is used to characterize the weight coefficient assigned to the driver in the human-machine collaborative control process, that is, the weight coefficient of the first control information.

[0039] Optionally, the delay information and response information are subjected to fuzzy control processing to obtain a first participation coefficient. The first participation coefficient is used to smooth the first allocation weight to avoid sudden changes in instructions and vehicle instability.

[0040] Optionally, the risk value of the vehicle is obtained based on the vehicle status information, and the second allocation weight is obtained based on the risk value of the vehicle. The second allocation weight is used to characterize the weight coefficient assigned to the vehicle in the human-machine collaborative control process, that is, the weight coefficient of the second control information.

[0041] Optionally, the vehicle state information is subjected to fuzzy control processing to obtain a second participation coefficient. The second participation coefficient is used to smooth the second allocation weight to avoid sudden changes in commands and vehicle instability.

[0042] Step S130: Determine the influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight.

[0043] Optionally, the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight are multiplied and added together to obtain the influence parameter.

[0044] Step S140: Based on the influence parameters, the first control information, and the second control information, determine the target control information for controlling the vehicle.

[0045] Optionally, the target control information for controlling the vehicle can be determined by weighted fusion of the first control information and the second control information based on the influence parameters.

[0046] It is understood that this invention decouples and analyzes multiple risk issues, including network latency represented by delay information, driver distraction represented by reaction information, and driving potential field risk represented by vehicle state information, to obtain driver and vehicle weights in the human-machine collaborative driving process. This enables multi-factor collaborative permission allocation, and then smooths the permissions of the first and second control information based on the allocated weights, avoiding risks caused by hard switching. This invention comprehensively considers the multiple risk factors existing in remote driving, achieving collaborative decision-making through multi-factor fusion, and solving the shortcomings of a single perspective, idealized technology, and incomplete analysis of factors. Furthermore, by fully utilizing the latency analysis and autonomous decision-making capabilities of the cloud system, the complex calculation process is transferred from the vehicle to the cloud edge computing unit, forming a collaborative enhanced intelligence that complements the advantages of the remote driver and intelligent vehicle, effectively improving driving safety and human-machine collaboration capabilities.

[0047] As an optional embodiment, determining a first participation coefficient and a first allocation weight based on the delay information and the response information, and determining a second participation coefficient and a second allocation weight based on the vehicle state information, includes: The delay information and the reaction information are respectively input into the first risk quantification model and the fuzzy logic model to obtain the first allocation weight output by the first risk quantification model and the first participation coefficient output by the fuzzy logic model. The vehicle status information is input into the second risk quantification model and the fuzzy logic model respectively to obtain the second allocation weight output by the second risk quantification model and the second participation coefficient output by the fuzzy logic model.

[0048] Optionally, the first risk quantification model is used to normalize the delay information and response information, and then the normalized delay information and response information are linearly summed to obtain the first allocation weight.

[0049] The latency information includes network latency and command issuance latency. Network latency includes parameters such as one-way latency, round-trip time, jitter, and packet loss rate, which describe the data transmission in the network from different dimensions and determine whether the vehicle can be controlled in a timely and stable manner. This embodiment of the invention uses multiple network latency parameters directly as core decision variables, adapting to fluctuations in network conditions.

[0050] Optionally, the vehicle status information includes potential field risk and tracking error. The vehicle status information is input into the second risk quantification model, that is, the normalized potential field risk index and tracking error index are input into the second risk quantification model. The second risk quantification model is an exponential model.

[0051] Optionally, the fuzzy logic model employs fuzzy set mapping rules to quantify delay information and response information to obtain the degree of delay response. After deriving and defuzzifying the degree of delay response, a first participation coefficient is obtained. It is also used to quantify the potential field risk index and tracking error index in vehicle state information to obtain the degree of potential field risk. After deriving and defuzzifying the degree of potential field risk, a second participation coefficient is obtained.

[0052] It is understood that this invention achieves dynamic allocation of permissions based on network latency awareness using latency information and response information, thereby improving adaptability and control robustness in network latency scenarios. It also achieves smooth continuity of permission allocation through a fuzzy logic model, avoiding sudden changes in control commands and vehicle instability caused by hard switching, and reducing the probability of accidents caused by sudden control jumps.

[0053] As an optional embodiment, the first risk quantification model is a neural network model or a linear model fitted by least squares, the second risk quantification model is a neural network model or an exponential model fitted by least squares, and the fuzzy logic model is obtained based on a neural network model, a Bayesian network model, or a linear fuzzy control model.

[0054] Optionally, the fuzzy logic model can be replaced by a neural network model or a Bayesian network model. In scenarios where computing resources are limited, a linear fuzzy control model can be used.

[0055] Optionally, both the first and second risk quantification models include two parts: risk analysis and numerical function models. The first risk quantification model can calculate the corresponding risk values ​​based on driving distraction and network latency, and calculate the first allocation weight based on the risk values. The second risk quantification model can calculate the corresponding risk value based on the potential field risk, and then calculate the second allocation weight based on the risk value. .

[0056] Optionally, the expression for the driver model that considers network latency information and reaction information is as follows: ; in, Let be the second derivative of the driver's front wheel steering angle. It is network latency information. It is the driver distraction time threshold, that is, the unacceptable time for the driver to be distracted. If the reaction information indicates that the driver is distracted for more than 10 seconds, the weight of the second control information on the vehicle side will be greatly increased. It is the driver's front wheel steering angle under delayed interference. These are driver characteristic parameters. It refers to the transmission ratio of the chassis. It is the steering gain. It is the first derivative of the driver's front wheel steering angle. It is the driver's aiming error.

[0057] In this embodiment of the invention, the driver's front wheel steering angle under the delay interference calculated by the above-mentioned driver model is uploaded to the cloud as the first control information, which can reduce the impact of time delay.

[0058] Based on the artificial potential field theory, this invention designs potential field risks and, taking into account the characteristics of complex working conditions, establishes accurate fuzzy numerical function models in sequence. Finally, it obtains a driver model that considers video stream transmission delay and driver distraction, thus solving the problems of traditional methods that consider only one factor and have inaccurate allocation.

[0059] Optionally, the expression for the first risk quantification model is as follows: ; in, , , , These are experimental parameters. It's a delay in issuing instructions. It is the value of the reaction information. This is network latency information.

[0060] Optionally, the expression for the second risk quantification model is as follows: ; in, , These are the potential field risk indicators. Tracking error index The parameters.

[0061] It is understood that the present invention obtains a first risk quantification model by fitting a neural network model or the least squares method, and obtains a second risk quantification model by fitting a neural network model or the least squares method. It obtains a fuzzy logic model based on a neural network model, a Bayesian network model or a linear fuzzy control model, which can be adapted to a variety of application scenarios and can also clearly analyze the impact of different factors on the allocation of human-machine permissions for remote driving.

[0062] As an optional embodiment, the fuzzy logic model is used to fuzzify the delay information and the response information, map the fuzzified delay information and response information based on fuzzy set mapping rules to obtain the delay response degree, and defuzzify the delay response degree to obtain the first participation coefficient; it is also used to fuzzify the potential field risk index and tracking error index in the vehicle state information, map the fuzzified potential field risk index and tracking error index based on fuzzy set mapping rules to obtain the potential field risk degree, and defuzzify the potential field risk degree to obtain the second participation coefficient.

[0063] Optional, , These are the first and second participation coefficients, respectively, derived from a fuzzy logic model. To analyze the impact of network latency, driver distraction, and vehicle driving risk, this invention proposes an online analysis and adjustment method for latency-dependent weights based on fuzzy logic. Figure 2 The inputs to the fuzzy logic model are total network delay, response information, potential field risk index, and tracking error index. First, the total network delay and response information are fuzzified, and the delay distraction degree (DL) is determined according to the fuzzy set mapping rule. After derivation and defuzzification, the first participation parameter is obtained. Simultaneously, the potential field risk index and the tracking error index are fuzzified, and the risk level (RL) is determined according to the corresponding fuzzy set mapping rule. The parameters are obtained through derivation and defuzzification. Specific parameters based on DL and RL and We design fuzzy set mapping rules and obtain participation coefficients through derivation and defuzzification. and This is used to calculate the final human-machine assigned permissions.

[0064] Figure 3 This is one of the schematic diagrams of fuzzy set mapping rules provided by the present invention. The total network latency Tsum is divided into low (L T ), low-medium (LM) T ), Middle (M) T ), Medium-high (HM) T ), High (H) T Level 5, its fuzzy set is defined as F1 ∈ {L} T LM T M T HM T H T}, membership functions such as Figure 3 As shown in (a). The reaction information d is divided into five levels: low (L... d ), low-medium (LM) d ), Middle (M) d ), Medium-high (HM) d ), high (H) d Its fuzzy set is defined as F2 ∈ {L d LM d M d HM d H d}, membership functions such as Figure 3 As shown in (b), the output λ is determined by Tsum and d. L1 Similarly, it is divided into five levels: low (L1), low-medium (LM1), medium (M1), medium-high (HM1), and high (H1), with its fuzzy set defined as O1∈ {L1, LM1, M1, HM1, H1}, and the membership function is shown in 3(c). The input variables Tsum and d are related to the output variable λ. L1 The three-dimensional relationship of fuzzy rules between them is as follows: Figure 3 As shown in (d).

[0065] Figure 4 This is the second schematic diagram of the fuzzy set mapping rule provided by the present invention, which involves input variable D. R Divided into low (L) DR ), low-medium (LM) DR ), Middle (M) DR ), medium to high (HM)DR ), high (H) DR Five levels, whose fuzzy set is defined as F DR ∈ {L DR LM DR M DR HM DR H DR}, membership functions such as Figure 4 As shown in (a). Input variable E R Divided into five levels: Low (L) ER ), low-medium (LM) ER ), Middle (M) ER ), medium to high (HM) ER ), high (H) ER Its fuzzy set is defined as F ER ∈ { L ER LM ER M ER HM ER H ER}, membership functions such as Figure 4 As shown in (b). By D R and E R Determined output λ L2 Similarly, it is divided into five levels: low (L2), low-medium (LM2), medium (M2), medium-high (HM2), and high (H2), with its fuzzy set defined as O2∈ {L2, LM2, M2, HM2, and H2}, and the membership function as follows: Figure 4 As shown in (c). Input variable D R and E R With output variable λ L2 The three-dimensional relationship of fuzzy rules between them is as follows: Figure 4 As shown in (d).

[0066] It is understood that this invention, based on the influencing factor adjustment mechanism of intelligent fuzzy control logic, establishes a fuzzy logic model, which is conducive to achieving smooth and continuous permission allocation, realizing fusion output based on fuzzy weights, avoiding the risk of sudden changes in control commands and vehicle instability caused by traditional hard switching, reducing the probability of accidents caused by sudden control jumps, and effectively avoiding the problem of chaotic permission allocation under different factors with different degrees of influence. It is also convenient to decouple and analyze different factors, realize the safety and stability of remote driving, and make full use of the intelligent vehicle's own efficient analysis and decision-making attributes.

[0067] As an optional embodiment, determining the influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight includes: Determine the first product between the first participation coefficient and the first allocation weight; Determine the second product between the second participation coefficient and the second allocation weight; The sum of the first product and the second product is used as the influence parameter; The step of determining the target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information includes: Using the influence parameters as weights, the first control information and the second control information are weighted and fused to obtain target control information for controlling the vehicle.

[0068] Optionally, the calculation formula for target control information is as follows: ; in, For target control information, This is the first control information. This is the second control information. To influence parameters.

[0069] It is understandable that this invention achieves collaborative decision-making through multi-factor fusion, and can simultaneously and comprehensively analyze multi-source risk information such as driver distraction, network latency, and potential field risk. It overcomes the problem of low accuracy in permission allocation caused by the single factor considered in traditional methods, and makes full use of the flexibility of human-cloud-vehicle collaborative control.

[0070] The following describes the human-machine collaborative control method and apparatus based on multi-source risk provided by the present invention. The human-machine collaborative control method and apparatus based on multi-source risk described below can be referred to in correspondence with the human-machine collaborative control method based on multi-source risk described above.

[0071] Figure 5 This is a schematic diagram of the human-machine collaborative control device based on multi-source risk provided by the present invention, as shown below. Figure 5 As shown, the present invention also provides a human-machine collaborative control device based on multi-source risk, comprising: The acquisition module 510 is used to acquire the first control information, delay information and response information sent by the operator terminal, and the vehicle status information and second control information sent by the vehicle terminal. The first determining module 520 is used to determine a first participation coefficient and a first allocation weight based on the delay information and the reaction information, and to determine a second participation coefficient and a second allocation weight based on the vehicle state information. The second determining module 530 is used to determine the influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight. The third determining module 540 is used to determine target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information.

[0072] As one embodiment, the first determining module 520 is configured to: The delay information and the reaction information are respectively input into the first risk quantification model and the fuzzy logic model to obtain the first allocation weight output by the first risk quantification model and the first participation coefficient output by the fuzzy logic model. The vehicle status information is input into the second risk quantification model and the fuzzy logic model respectively to obtain the second allocation weight output by the second risk quantification model and the second participation coefficient output by the fuzzy logic model.

[0073] As an example, the first risk quantification model is a neural network model or a linear model fitted by the least squares method, the second risk quantification model is an exponential model fitted by the neural network model or the least squares method, and the fuzzy logic model is obtained based on a neural network model, a Bayesian network model, or a linear fuzzy control model.

[0074] As an example, the fuzzy logic model is used to fuzzify the delay information and the response information, map the fuzzified delay information and response information based on fuzzy set mapping rules to obtain the degree of delay response, and defuzzify the degree of delay response to obtain the first participation coefficient; it is also used to fuzzify the potential field risk index and the tracking error index in the vehicle state information, map the fuzzified potential field risk index and the tracking error index based on fuzzy set mapping rules to obtain the degree of potential field risk, and defuzzify the degree of potential field risk to obtain the second participation coefficient.

[0075] As one embodiment, the second determining module 530 is used for: Determine the first product between the first participation coefficient and the first allocation weight; Determine the second product between the second participation coefficient and the second allocation weight; The sum of the first product and the second product is used as the influence parameter; The third determining module 540 is used for: Using the influence parameters as weights, the first control information and the second control information are weighted and fused to obtain target control information for controlling the vehicle.

[0076] It should be noted that the human-machine collaborative control method and device based on multi-source risk provided by the present invention has the same technical effects as the human-machine collaborative control method based on multi-source risk described above, and will not be repeated here.

[0077] The human-machine collaborative control method system based on multi-source risk provided by the present invention is described below. The human-machine collaborative control method system based on multi-source risk described below can be referred to in correspondence with the human-machine collaborative control method based on multi-source risk described above.

[0078] Figure 6 This is a schematic diagram of the human-machine collaborative control system based on multi-source risk provided by the present invention, as shown below. Figure 6 As shown, this invention also provides a human-machine collaborative control system based on multi-source risks, including a cloud, an operator terminal, and a vehicle terminal. There is a video stream transmission delay between the operator terminal and the cloud, and a command issuance delay and a vehicle status feedback delay between the cloud and the vehicle terminal. By using these three different network delays as decision variables for permission allocation, a breakthrough is achieved from passively adapting to delays to actively allocating permissions based on delays.

[0079] The cloud platform is used to acquire first control information, delay information, and response information sent by the operator terminal, as well as vehicle status information and second control information sent by the vehicle terminal; based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle status information, a second participation coefficient and a second allocation weight are determined; based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, an influence parameter is determined; based on the influence parameter, the first control information, and the second control information, target control information for controlling the vehicle is determined.

[0080] As an optional embodiment, the vehicle end is equipped with a model prediction controller, which is used to acquire the vehicle state information and determine the second control information based on the vehicle state information to complete the forward roll optimization of the vehicle.

[0081] As an optional embodiment, the operator terminal is equipped with a driver model, which is used to determine the first control information based on the delay information and the reaction information.

[0082] As an optional embodiment, the cloud is also used for: The delay information and the reaction information are respectively input into the first risk quantification model and the fuzzy logic model to obtain the first allocation weight output by the first risk quantification model and the first participation coefficient output by the fuzzy logic model. The vehicle status information is input into the second risk quantification model and the fuzzy logic model respectively to obtain the second allocation weight output by the second risk quantification model and the second participation coefficient output by the fuzzy logic model.

[0083] As an optional embodiment, the first risk quantification model is a neural network model or a linear model fitted by least squares, the second risk quantification model is a neural network model or an exponential model fitted by least squares, and the fuzzy logic model is obtained based on a neural network model, a Bayesian network model, or a linear fuzzy control model.

[0084] As an optional embodiment, the fuzzy logic model is used to fuzzify the delay information and the response information, map the fuzzified delay information and response information based on fuzzy set mapping rules to obtain the delay response degree, and defuzzify the delay response degree to obtain the first participation coefficient; it is also used to fuzzify the potential field risk index and tracking error index in the vehicle state information, map the fuzzified potential field risk index and tracking error index based on fuzzy set mapping rules to obtain the potential field risk degree, and defuzzify the potential field risk degree to obtain the second participation coefficient.

[0085] As an optional embodiment, the cloud is also used for: Determine the first product between the first participation coefficient and the first allocation weight; Determine the second product between the second participation coefficient and the second allocation weight; The sum of the first product and the second product is used as the influence parameter; The step of determining the target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information includes: Using the influence parameters as weights, the first control information and the second control information are weighted and fused to obtain target control information for controlling the vehicle.

[0086] In summary, this application proposes a human-machine collaborative shared control permission allocation method based on multi-source risk to address multi-factor, multi-source risk problems. This method introduces human-machine shared control into a comprehensive analysis scenario involving remote driving distraction, network latency, and driving potential field risks. Furthermore, this invention analyzes various complex operating conditions, improving the trajectory tracking accuracy of remote driving human-machine shared control. Under extreme and dangerous conditions of driving distraction, the weight on the vehicle side is flexibly and instantaneously increased, avoiding the abrupt switching drawbacks of traditional switching control and reducing road risks.

[0087] Figure 7An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a human-machine collaborative control method based on multi-source risks, the method including: Acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal; Based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle state information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, the influencing parameters are determined; Based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined.

[0088] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the human-machine collaborative control method based on multi-source risk provided by the above methods, the method comprising: Acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal; Based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle state information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, the influencing parameters are determined; Based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined.

[0090] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the human-machine collaborative control method based on multi-source risk provided by the above methods, the method comprising: Acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal; Based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle state information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, the influencing parameters are determined; Based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human-machine collaborative control method based on multi-source risk, characterized in that, Applied to the cloud, including: Acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal; Based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle state information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, the influencing parameters are determined; Based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined.

2. The human-machine collaborative control method based on multi-source risk according to claim 1, characterized in that, The step of determining a first participation coefficient and a first allocation weight based on the delay information and the response information, and determining a second participation coefficient and a second allocation weight based on the vehicle state information, includes: The delay information and the reaction information are respectively input into the first risk quantification model and the fuzzy logic model to obtain the first allocation weight output by the first risk quantification model and the first participation coefficient output by the fuzzy logic model. The vehicle status information is input into the second risk quantification model and the fuzzy logic model respectively to obtain the second allocation weight output by the second risk quantification model and the second participation coefficient output by the fuzzy logic model.

3. The human-machine collaborative control method based on multi-source risk according to claim 2, characterized in that, The first risk quantification model is a neural network model or a linear model obtained by least squares fitting, the second risk quantification model is a neural network model or an exponential model obtained by least squares fitting, and the fuzzy logic model is obtained based on a neural network model, a Bayesian network model or a linear fuzzy control model.

4. The human-machine collaborative control method based on multi-source risk according to claim 2, characterized in that, The fuzzy logic model is used to fuzzify the delay information and the response information, and to map the fuzzified delay information and response information based on the fuzzy set mapping rule to obtain the degree of delayed response. The degree of delayed response is then defuzzified to obtain the first participation coefficient. It is also used to fuzzify the potential field risk index and tracking error index in the vehicle state information, map the fuzzified potential field risk index and tracking error index based on the fuzzy set mapping rule to obtain the potential field risk level, and defuzzify the potential field risk level to obtain the second participation coefficient.

5. The human-machine collaborative control method based on multi-source risk according to any one of claims 1 to 4, characterized in that, The determination of influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight includes: Determine the first product between the first participation coefficient and the first allocation weight; Determine the second product between the second participation coefficient and the second allocation weight; The sum of the first product and the second product is used as the influence parameter; The step of determining the target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information includes: Using the influence parameters as weights, the first control information and the second control information are weighted and fused to obtain target control information for controlling the vehicle.

6. A human-machine collaborative control device based on multi-source risk, characterized in that, include: The acquisition module is used to acquire the first control information, delay information, and response information sent by the operator terminal, as well as the vehicle status information and second control information sent by the vehicle terminal. The first determining module is used to determine a first participation coefficient and a first allocation weight based on the delay information and the response information, and to determine a second participation coefficient and a second allocation weight based on the vehicle state information. The second determining module is used to determine the influencing parameters based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight. The third determining module is used to determine target control information for controlling the vehicle based on the influence parameters, the first control information, and the second control information.

7. A human-machine collaborative control system based on multi-source risk, implemented based on the human-machine collaborative control method based on multi-source risk as described in any one of claims 1 to 5, characterized in that, This includes the cloud, operator terminals, and vehicle terminals; The cloud is used to acquire first control information, delay information, and response information sent by the operator terminal, as well as vehicle status information and second control information sent by the vehicle terminal. Based on the delay information and the response information, a first participation coefficient and a first allocation weight are determined, and based on the vehicle state information, a second participation coefficient and a second allocation weight are determined. Based on the first participation coefficient, the first allocation weight, the second participation coefficient, and the second allocation weight, influence parameters are determined; based on the influence parameters, the first control information, and the second control information, target control information for controlling the vehicle is determined. The vehicle is equipped with a model prediction controller, which is used to acquire the vehicle state information and determine the second control information based on the vehicle state information to complete the forward roll optimization of the vehicle. The operator terminal is equipped with a driver model, which is used to determine the first control information based on the delay information and the reaction information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the human-machine collaborative control method based on multi-source risk as described in any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the human-machine collaborative control method based on multi-source risk as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the human-machine collaborative control method based on multi-source risk as described in any one of claims 1 to 5.

Images