Human-machine cooperation method and device for networked automobile based on YTS, and electronic equipment

Abstract

The invention provides a man-machine cooperation method and device for a networked automobile based on a YTS and electronic equipment, relates to the technical field of data processing, and solves the technical problem that the execution operation of man-machine cooperation of a vehicle is low in effectiveness at present. The method comprises the following steps: acquiring first environment data around a networked automobile through a vehicle-mounted sensor; determining to-be-executed operation based on the first environment data and a control instruction triggered by a user corresponding to the networked automobile; sending the first environment data to a cloud, simulating the first environment data at the cloud based on a YTS engine, and when execution time corresponding to the to-be-executed operation does not exceed a specified deadline, applying the to-be-executed operation in a simulation environment of the YTS engine to obtain an execution result; and optimizing the to-be-executed operation based on the execution result to obtain an optimized to-be-executed operation, and sending the optimized to-be-executed operation to the networked automobile.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a man-machine cooperation method and device for a YTS-based connected vehicle and electronic equipment. BACKGROUND

[0002] At present, with the development of 5G communication technology and automatic driving technology, intelligent connected vehicles gradually evolve from auxiliary driving to higher-order man-machine collaborative driving. The current mainstream man-machine cooperation system mainly adopts a hybrid architecture of local computing and cloud collaboration. According to statistical data, about 65% of traffic accidents are caused by the misjudgment of drivers in complex traffic scenes. The existing collaborative decision-making system has a decision-making accuracy rate of less than 80% when dealing with edge cases such as "ghost probe" and "emergency lane change". This exposes the shortcomings of traditional methods in dynamic environment adaptability and decision-making confidence assessment. Therefore, the effectiveness of the execution operation of the current man-machine cooperation for vehicles is low. SUMMARY

[0003] The purpose of the present application is to provide a man-machine cooperation method and device for a YTS-based connected vehicle and electronic equipment to solve the technical problem of low effectiveness of the execution operation of the current man-machine cooperation for vehicles.

[0004] In a first aspect, the present application provides a man-machine cooperation method for a YTS-based connected vehicle, wherein a vehicle-mounted sensor is arranged on the connected vehicle, and the method comprises: obtaining first environment data around the connected vehicle through the vehicle-mounted sensor; determining a to-be-executed operation based on the first environment data and a control instruction triggered by a user corresponding to the connected vehicle, wherein the to-be-executed operation corresponds to an execution time; sending the first environment data to the cloud, simulating the first environment data based on a YTS engine in the cloud, and applying the to-be-executed operation in a simulation environment of the YTS engine when the execution time corresponding to the to-be-executed operation does not exceed a specified deadline to obtain an execution result, wherein the simulation environment is obtained by fusion simulation based on the first environment data and second environment data of other vehicles related to the location of the connected vehicle; optimizing the to-be-executed operation based on the execution result to obtain an optimized to-be-executed operation, and sending the optimized to-be-executed operation to the connected vehicle; verifying the execution time of the optimized to-be-executed operation, and executing the optimized to-be-executed operation at the corresponding verified time.

[0005] In one possible implementation, the construction process of the simulation environment in the cloud comprises: The cloud receives second environmental data from other vehicles within the geographical fence area where the connected vehicle is located; Based on the first environmental data and the second environmental data, a spatiotemporal alignment algorithm is used to fuse multi-source heterogeneous environmental data into a unified coordinate system, and a dynamic traffic situation model is established based on the fused environmental data in the unified coordinate system.

[0006] In one possible implementation, the step of fusing multi-source heterogeneous environmental data into a unified coordinate system based on the first environmental data and the second environmental data using a spatiotemporal alignment algorithm, and establishing a dynamic traffic situation model based on the fused environmental data in the unified coordinate system, includes: The first environmental data and the second environmental data are mapped to a unified time and space coordinate system by affine transformation to obtain first environmental mapping data and second environmental mapping data; wherein, the first environmental data or the second environmental data includes at least three of the following: obstacle location data, road sign recognition results, meteorological monitoring data, traffic light status, and adjacent vehicle driving trajectories. Based on the first and second environment mapping data, a spatiotemporal alignment algorithm is used to fuse multi-source heterogeneous environment data into a unified coordinate system using the following formula: F ( p , t )= w 1 D 1( p , t )+ w 2 D 2( p , t );in, w 1 and w 2 is the weighting coefficient. D 1( p , t ) represents a specific location in the first environment mapping data. p In time t Data, D 1( p , t ) represents a specific location in the second environment mapping data. p In time t Data, F ( p , t ) represents the value obtained by fusing the first environment mapping data and the second environment mapping data; Based on the fused values, a dynamic traffic situation model is established using a fluid dynamics model. The dynamic traffic situation model includes at least one of traffic flow parameters, speed distribution parameters, and congestion parameters.

[0007] In a possible implementation, the fusion of the multi-source heterogeneous environment data into a unified coordinate system based on the first environment data and the second environment data is performed by a space-time alignment algorithm, including: The fusion of the multi-source heterogeneous environment data into a unified coordinate system based on the first environment data and the second environment data is performed by using an improved Kalman-Filter space-time fusion algorithm according to the following formula: k ∣ k = k -1∣ k -1+ uk ; wherein, Δ t is a time difference between the first environment data and the second environment data, is a unit matrix corresponding to the first environment data and the second environment data, uk is a quantity of a control input corresponding to the control instruction, k -1∣ k -1 represents state estimation data before fusion at a previous moment, k ∣ k represents state estimation data after fusion at a current moment.

[0008] In a possible implementation, the optimization of the to-be-executed operation based on the execution result includes: iterative optimization of operation parameters corresponding to the to-be-executed operation is performed by using a reinforcement learning algorithm to obtain an iterative optimization result; an optimized trajectory is generated based on the iterative optimization result and a preset safety boundary constraint condition, and operation feasibility of the optimized trajectory is verified by using a digital twin method to obtain an operation verification result, and the optimized to-be-executed operation is obtained based on the operation verification result.

[0009] In a possible implementation, the verification of the execution time of the optimized to-be-executed operation includes: a timestamp verification module is established, and the timestamp verification module is used to synchronize a cloud system clock and a vehicle-mounted system clock by using an NTP protocol to obtain a time synchronization result; time window matching degree detection and remaining validity period calculation are performed based on the time synchronization result before operation execution to obtain a time detection result; if the time detection result meets specified time verification index data, it is determined that a time verification result of the optimized to-be-executed operation is verified.

[0010] In a possible implementation, the simulation environment in the cloud comprises a three-dimensional road network digital twin built based on the YTS engine, and integrates a V2X communication protocol stack simulator, a traffic flow microscopic simulation module, and a vehicle dynamics model.

[0011] In a possible implementation, the sending of the optimized to-be-executed operation to the connected car comprises: differentially encrypting an operation instruction corresponding to the optimized to-be-executed operation to obtain a differential encryption result; establishing an MQTT protocol transmission channel, sending the differential encryption result to the connected car through the MQTT protocol transmission channel, and recording an operation log corresponding to the operation instruction through a block chain evidence recording manner.

[0012] In a second aspect, the present application provides a man-machine cooperation device for a connected car based on YTS, wherein the connected car is provided with a vehicle-mounted sensor, and the device comprises: an acquisition module configured to acquire first environment data around the connected car through the vehicle-mounted sensor; a determination module configured to determine a to-be-executed operation based on the first environment data and a control instruction triggered by a user corresponding to the connected car, the to-be-executed operation corresponding to an execution time; a simulation module configured to send the first environment data to a cloud, simulate the first environment data based on a YTS engine in the cloud, and apply the to-be-executed operation in a simulation environment of the YTS engine when the execution time corresponding to the to-be-executed operation does not exceed a specified deadline to obtain an execution result, wherein the simulation environment is obtained through fusion simulation based on the first environment data and second environment data of other vehicles related to a location of the connected car; an optimization module configured to optimize the to-be-executed operation based on the execution result to obtain an optimized to-be-executed operation, and send the optimized to-be-executed operation to the connected car; an execution module configured to verify an execution time of the optimized to-be-executed operation, and execute the optimized to-be-executed operation at a time corresponding to the verification.

[0013] In a third aspect, the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor implements the method of the first aspect when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer readable storage medium, which stores computer executable instructions. When the computer executable instructions are called and executed by a processor, the computer executable instructions cause the processor to execute the method of the first aspect.

[0015] The present application brings the following beneficial effects: The man-machine cooperation method and device based on YTS for a connected vehicle and the electronic device provided by the present application can obtain first environment data around the connected vehicle through a vehicle-mounted sensor arranged on the connected vehicle, determine a to-be-executed operation based on the first environment data and a control instruction triggered by a user corresponding to the connected vehicle, the to-be-executed operation corresponding to an execution time, send the first environment data to a cloud, and simulate the first environment data based on a YTS engine at the cloud, and when the execution time corresponding to the to-be-executed operation does not exceed a specified deadline, apply the to-be-executed operation in a simulation environment of the YTS engine to obtain an execution result, wherein the simulation environment is obtained by fusion simulation based on the first environment data and second environment data of other vehicles related to a location of the connected vehicle, optimize the to-be-executed operation based on the execution result to obtain an optimized to-be-executed operation, and send the optimized to-be-executed operation to the connected vehicle, verify the execution time of the optimized to-be-executed operation, and execute the optimized to-be-executed operation at a corresponding verified time. In the present application, the cloud simulation verification mechanism of the YTS engine allows different operation strategies to be tested in a virtual environment, ensures that the optimal operation scheme is selected, and this method not only considers the instant environment condition, but also simulates possible future changes, so that more accurate and forward-looking decisions can be made. Not only real-time simulation verification is realized, but also dynamic environment fusion of the simulation environment is realized. When an extreme working condition that has not been trained appears, the system can also make reasonable execution operation decisions, thereby enhancing the decision accuracy of the execution operation and improving the effectiveness of the execution operation of the man-machine cooperation for the vehicle.

[0016] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0018] Figure 1 A flowchart of a man-machine cooperation method for a connected vehicle based on YTS is provided in an embodiment of the present application. Figure 2 Another flowchart of a man-machine cooperation method for a connected vehicle based on YTS is provided in an embodiment of the present application. Figure 3 A structural diagram of a man-machine cooperation device for a connected vehicle based on YTS is provided in an embodiment of the present application. Figure 4 A structural diagram of an electronic device is shown in an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in detail with reference to the drawings, obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0020] The terms “include” and “have” and any variations thereof mentioned in the embodiments of the present application are intended to cover the inclusions without exclusivity. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0021] At present, the development of vehicle-to-everything (V2X) technology provides new possibilities for intelligent driving. From the early Telematics system to the current C-V2X communication, the information interaction ability between vehicles and the outside world is continuously enhanced. The third generation V2X standard supports 5G NR-V2X direct communication, with a delay as low as 1ms and a bandwidth of 1Gbps, which provides a technical basis for real-time simulation verification. However, the existing technology still has obvious defects in the aspect of cooperative decision-making: most systems adopt a centralized decision-making architecture, relying on roadside units (RSUs) for global scheduling, which has the risk of single-point failure; although some distributed solutions improve reliability, they lack a unified simulation verification mechanism, making it difficult to ensure decision consistency.

[0022] In practical applications, intelligent networked vehicles face three major technical bottlenecks: first, vehicle-mounted sensors are difficult to achieve 360° all-around environment perception due to the limitations of physical installation location and hardware performance, especially in adverse weather conditions such as heavy rain and thick fog, the perception accuracy will decrease significantly; second, the iteration period of traditional local decision-making algorithms is long, which cannot respond to complex and variable traffic scenarios in real time, for example, the emergency obstacle avoidance decision at the intersection often requires a response time of seconds; finally, the existing vehicle networking system lacks effective multi-vehicle coordination mechanism, and independent decision-making of each vehicle can easily lead to low traffic flow efficiency, according to statistics, the traffic delay caused by insufficient vehicle coordination accounts for up to 27% of urban road traffic.

[0023] In the prior art, some solutions attempt to introduce cloud computing resources, but due to network delay and data security restrictions, it is difficult to achieve real-time simulation verification. For example, a remote driving solution adopted by a certain vehicle manufacturer has a control instruction transmission delay of more than 200ms, which cannot meet the requirement of ISO 26262 functional safety standard that the decision response time should be less than or equal to 100ms. In addition, the existing system constructs the simulation environment by using a pre-set scene library, which lacks dynamic environment fusion capability, and when extreme conditions that have not been trained appear, the system often cannot make reasonable decisions. Therefore, the current execution operation effectiveness of human-machine collaboration for vehicles is low.

[0024] Based on this, the embodiments of the present application provide a YTS-based human-machine collaboration method, device and electronic equipment for networked vehicles, which can solve the technical problem of low execution operation effectiveness of human-machine collaboration for vehicles.

[0025] The embodiments of the present application will be further described below with reference to the accompanying drawings.

[0026] Figure 1 A flowchart of a YTS-based human-machine collaboration method for networked vehicles is provided. The networked vehicle is provided with a vehicle-mounted sensor. As shown in Figure 1 The method comprises the following steps: Step S110, acquiring first environment data around the networked vehicle through the vehicle-mounted sensor.

[0027] As an optional implementation, the method constructs a layered architecture system consisting of four layers: a perception layer: integrates millimeter wave radar (77GHz), solid-state laser radar (128 lines), 8 million pixel surround view camera (120° FOV) and other sensors, and the sampling frequency reaches 200Hz; a transmission layer: adopts 5G NR-V2X PC5 direct connection communication, the delay is controlled within 3ms, and supports vehicle networking within a radius of 1000 meters; a simulation layer: a digital twin is constructed based on YTS engine, including a high-precision road model (the precision reaches centimeter level) and a dynamic traffic element library (supports more than 100,000 entity objects); a decision layer: a federal learning framework is deployed, and each node keeps local data privacy while optimizing the decision model cooperatively For the data collection of the above vehicle-mounted sensor, the environment data is collected in real time through the vehicle-mounted sensor network. For example, dynamic obstacle detection: a YOLOv5s algorithm is used to realize 100ms level target detection, and a bounding box with a confidence level of >95% is output; road state perception: a point cloud registration algorithm is used to generate road curvature, slope and other parameters, and the precision reaches ±0.1m / km; meteorological data fusion: meteorological satellite data and vehicle-mounted meteorological station information are received, and a multivariate Kalman filtering model is established. Then, the data is preprocessed, and the preprocessing process includes: time synchronization: PTP protocol is used to realize sub-microsecond time alignment; spatial registration: GNSS-RTK positioning data (horizontal accuracy ±10cm) is used for coordinate conversion; feature extraction: PointNet++ network is used to extract point cloud features, and the dimension is reduced to 128-dimensional feature vectors.

[0028] Step S120, based on the first environment data and the control instruction triggered by the user corresponding to the connected car, the operation to be executed is determined.

[0029] Among them, the operation to be executed corresponds to an execution time. For example, the system can also determine the execution time corresponding to the specific operation to be executed when determining the specific operation to be executed.

[0030] For the processing of the user's control instruction reception, exemplary: the system receives control instructions from the user. This can be achieved through the vehicle entertainment information system, smart phone application or other interactive ways. User instructions may include acceleration, deceleration, steering, parking and other basic operation requests. Then, the first environmental data obtained from various sensors is integrated, and combined with the current state of the vehicle (such as speed, direction, etc.), a comprehensive understanding of the environment around the vehicle is formed, and the control instruction provided by the user is analyzed to understand the operation goal the user wants to achieve. Then, based on the fused environmental data and user instructions, the safety of executing the user's request is evaluated. If potential risks are found (for example, there are obstacles in front of the user and the user requests to accelerate), the instruction needs to be adjusted or refused to execute. Of course, it is also necessary to determine the specific operation to be executed and the corresponding execution time. Consider how to meet the user's needs while ensuring safety. For example, calculate the most appropriate steering angle, acceleration and other parameters. For the execution of the operation, the operation is executed according to the predetermined plan. This may involve sending corresponding control signals to the vehicle's power system, braking system, steering system.

[0031] Step S130, the first environmental data is sent to the cloud, and the first environmental data is simulated based on the YTS engine in the cloud, and when the execution time of the to-be-executed operation does not exceed the specified deadline, the to-be-executed operation is applied in the simulation environment of the YTS engine to obtain the execution result.

[0032] Among them, the simulation environment is obtained by simulating and fusing the first environmental data and the second environmental data of other vehicles related to the location of the connected car. Exemplary, the simulation environment in the cloud includes a three-dimensional road network digital twin body constructed based on the YTS engine, and integrates a V2X communication protocol stack simulator, a traffic flow micro-simulation module and a vehicle dynamics model. By combining vehicle dynamics modeling, V2X communication simulation, traffic flow micro-simulation module and three-dimensional road network digital twin body, the cloud simulation engine can run more efficiently.

[0033] It should be noted that the YTS (Unity TV Service) engine represents the unity visual rendering service system. Unity is a real-time 3D interactive content creation and operation platform, including game development, art, architecture, car design, film and television, and all creators who turn ideas into reality with unity. The platform provides a complete software solution for creating, operating and monetizing any real-time interactive 2D and 3D content, and supports platforms including mobile phones, tablets, PCs, game consoles, augmented reality and virtual reality devices. The YTS engine is an intelligent engine system deeply integrated with AI algorithms, physical simulation and 3D digital rendering technology, designed for the next generation of intelligent vehicles. Its core goal is to promote the overall upgrade of vehicle automation, vehicle-road cooperation, intelligent interaction and other fields through high-precision simulation, real-time decision optimization and cross-domain collaboration capabilities, and to build an integrated intelligent transportation ecosystem of "people-vehicles-roads-cloud".

[0034] The YTS (Unity TV Service) engine is an extension module of the Unity real-time 3D platform, designed and developed for intelligent transportation systems. The engine integrates Unity's HDRP (High Definition Rendering Pipeline) and DOTS (Data-Oriented Technology Stack), with ultra-strong rendering capabilities of processing millions of polygons per second, while supporting real-time simulation of physical engines. In the automotive industry, the YTS engine has been successfully applied in virtual test drive, collision simulation and production line layout optimization. For example, BMW Group used the YTS engine to build a digital twin factory, achieving a significant 15% improvement in assembly line efficiency. Compared to traditional simulation tools, the YTS engine has three major technical advantages: first, based on the ECS (Entity Component System) architecture, it can achieve millisecond-level scene switching and dynamic parameter adjustment; second, the built-in DOTS physics system can accurately simulate vehicle dynamics, including tire slip rate, suspension deformation and aerodynamic effects; third, it supports multi-platform collaborative simulation, integrating device data from different brands and different communication protocols in the same virtual environment. These features make it an ideal platform for building intelligent connected vehicle human-machine collaboration systems.

[0035] In an optional implementation, as shown in Figure 2 The construction process of the simulation environment in the cloud can include the following steps: Step S210, the cloud receives second environment data from other vehicles within the geographic fence range of the connected vehicle; Step S220, based on the first environment data and the second environment data, the multi-source heterogeneous environment data is fused into a unified coordinate system through a space-time alignment algorithm; Step S230, based on the fused environment data in the unified coordinate system, a dynamic traffic situation model is established.

[0036] The multi-source heterogeneous environment data is fused into a unified coordinate system by the first environment data and the second environment data using the space-time alignment algorithm, and a dynamic traffic situation model is established based on the fused environment data in the unified coordinate system, so that the data accuracy of the constructed dynamic traffic situation model is higher.

[0037] As an example, the step S130 of fusing the multi-source heterogeneous environment data into the unified coordinate system based on the first environment data and the second environment data using the space-time alignment algorithm, and establishing the dynamic traffic situation model based on the fused environment data in the unified coordinate system can specifically include the following steps: The first environment data and the second environment data are mapped into a unified time and space coordinate system by an affine transformation method to obtain first environment mapping data and second environment mapping data; wherein the first environment data or the second environment data includes at least three of obstacle position data, road sign recognition result, weather monitoring data, traffic signal light state and adjacent vehicle driving trajectory; Based on the first environment mapping data and the second environment mapping data, the multi-source heterogeneous environment data is fused into a unified coordinate system by using a space-time alignment algorithm through the following formula: F(p,t)=w1D1(p,t)+w2D2(p,t); wherein w1 and w2 are weight coefficients, D1(p,t) represents the data of a specific position p in the first environment mapping data at time t, D2(p,t) represents the data of the specific position p in the second environment mapping data at time t, and F(p,t) represents the corresponding fused value of the first environment mapping data and the second environment mapping data; a dynamic traffic situation model is established based on the fused value using a fluid mechanics model, and the dynamic traffic situation model includes at least one of traffic flow parameter, speed distribution parameter and congestion situation parameter.

[0038] In the embodiment of the application, the data accuracy of the finally constructed dynamic traffic situation model is further improved by using the space-time alignment algorithm through the above formula.

[0039] As another example, the step S130 of fusing the multi-source heterogeneous environment data into the unified coordinate system based on the first environment data and the second environment data using the space-time alignment algorithm can specifically include the following steps: Based on the first environment data and the second environment data, the multi-source heterogeneous environment data is fused into a unified coordinate system by using an improved Kalman-Filter space-time fusion algorithm through the following formula: k ∣ k = k -1∣ k -1+ uk ; wherein, Δ t is a time difference between the first environment data and the second environment data, is a unit matrix corresponding to the first environment data and the second environment data, uk is a quantity of the control input corresponding to the control instruction, k -1| x k-1 k -1 represents the pre-fusion state estimation data at the last time point, k | x k k represents the post-fusion state estimation data at the current time point.

[0040] In the embodiments of the present application, the improved Kalman-Filter space-time fusion algorithm can further improve the data accuracy of the post-fusion state estimation data.

[0041] In step S140, the to-be-executed operation is optimized based on the execution result to obtain an optimized to-be-executed operation, and the optimized to-be-executed operation is sent to the connected car.

[0042] In an optional embodiment, in step S140, the to-be-executed operation is optimized based on the execution result to obtain an optimized to-be-executed operation, which can specifically include the following steps: iteratively optimizing the operation parameters corresponding to the to-be-executed operation by using a reinforcement learning algorithm to obtain an iterative optimization result; generating an optimized trajectory based on the iterative optimization result and a preset safety boundary constraint condition, verifying the operation feasibility of the optimized trajectory by a digital twin method to obtain an operation verification result, and obtaining the optimized to-be-executed operation based on the operation verification result.

[0043] For the simulation verification stage, for example, a digital twin scene is loaded in the YTS engine, sensor simulation parameters (such as a laser radar noise model: Gaussian distribution, σ=0.05m) are set, and Monte Carlo simulation is run, with a sample size of no less than By the above data processing method, the data of the to-be-executed operation can be further optimized more accurately.

[0044] As an optional embodiment, in step S140, the optimized to-be-executed operation is sent to the connected car, which can specifically include the following steps: performing differential encryption processing on the operation instruction corresponding to the optimized to-be-executed operation to obtain a differential encryption result; establishing an MQTT protocol transmission channel, sending the differential encryption result to the connected car through the MQTT protocol transmission channel, and recording the operation log corresponding to the operation instruction by a blockchain storage method. By this data processing method, the transmission efficiency and data security of the to-be-executed operation can be further improved.

[0045] Step S150, the execution time of the optimized to-be-executed operation is verified, and the optimized to-be-executed operation is executed at the corresponding verified time.

[0046] In a possible implementation, the verification of the execution time of the optimized to-be-executed operation in step S150 can specifically include the following steps: establishing a time stamp verification module, and synchronizing the cloud and the vehicle-mounted system clock through the NTP protocol by using the time stamp verification module to obtain a time synchronization result; performing time window matching degree detection and remaining validity period calculation based on the time synchronization result before operation execution to obtain a time detection result; and if the time detection result meets the specified time verification index data, determining that the time verification result of the optimized to-be-executed operation is verified. Through this data processing manner, the verification efficiency of the to-be-executed operation can be improved.

[0047] For example, the clock source module includes a main clock and a backup clock. The main clock uses a GPS-tamed OCXO (temperature-compensated crystal oscillator) with a daily stability of ±5e-11, and the backup clock uses a temperature-compensated crystal oscillator (TCXO) with a daily drift of ±1e-8. For the management of time stamps, the Hybrid Logical Clocks (HLC) algorithm is used to combine physical clocks and logical clocks, and the time stamp resolution reaches 1 ns, and the sequence number space is 64 bits.

[0048] In the embodiment of the application, the cloud simulation verification mechanism of the YTS engine allows different operation strategies to be tested in a virtual environment, ensuring that the optimal operation scheme is selected. This method not only considers the current environmental conditions, but also simulates possible future changes, thereby making more accurate and forward-looking decisions. Not only is real-time simulation verification achieved, but dynamic environment fusion of the simulation environment is also achieved. When an untrained extreme working condition occurs, the system can also make reasonable execution operation decisions, enhancing the decision accuracy of the execution operation and thereby improving the effectiveness of the human-machine cooperation execution operation for the vehicle.

[0049] Moreover, the combination of the first environment data obtained by the vehicle-mounted sensor and the second environment data of other vehicles obtained from the cloud enables comprehensive evaluation of the safety of the to-be-executed operation in the simulation environment, which enables the system to identify potential risks before execution and make corresponding adjustments or optimizations. Furthermore, by quickly sending the preliminarily determined to-be-executed operation to the cloud for simulation optimization and rapidly returning the optimization result, the entire process can significantly reduce decision-making time, speed up response speed, and improve response speed and efficiency, which is particularly important for situations that require quick reactions (such as emergency braking).

[0050] For example, in a highway scenario, the system decision accuracy is improved from 82% of the traditional method to 96%, the multi-vehicle coordination communication overhead is reduced by 40%, the user takeover request frequency is reduced by 75%, the effective response time in emergency is shortened to 0.3 seconds, and the system average failure-free operation time reaches 12000 hours. The case verifies the effectiveness of the method in complex urban scenarios, especially in typical scenarios such as intersection coordination, emergency vehicle avoidance, etc., which fully meets the functional safety requirements of ISO 26262 and SAE J3016.

[0051] Figure 3 A structural schematic diagram of a YTS-based man-machine cooperation device of a connected vehicle is provided, and a vehicle-mounted sensor is arranged on the connected vehicle. Figure 3 As shown in the figure, the YTS-based man-machine cooperation device 300 of the connected vehicle comprises: An acquisition module 301 is configured to acquire first environment data around the connected vehicle through the vehicle-mounted sensor; A determination module 302 is configured to determine a to-be-executed operation based on the first environment data and a control instruction triggered by a user corresponding to the connected vehicle, the to-be-executed operation corresponding to an execution time; An emulation module 303 is configured to send the first environment data to a cloud end, perform emulation on the first environment data based on a YTS engine at the cloud end, and apply the to-be-executed operation in a simulation environment of the YTS engine when the execution time corresponding to the to-be-executed operation does not exceed a specified deadline to obtain an execution result, wherein the simulation environment is obtained by fusion emulation based on the first environment data and second environment data of other vehicles related to the location of the connected vehicle; An optimization module 304 is configured to optimize the to-be-executed operation based on the execution result to obtain an optimized to-be-executed operation, and send the optimized to-be-executed operation to the connected vehicle; An execution module 305 is configured to verify the execution time of the optimized to-be-executed operation, and execute the optimized to-be-executed operation at the corresponding verified time.

[0052] The YTS-based man-machine cooperation device of the connected vehicle provided by the embodiments of the present application has the same technical features as the YTS-based man-machine cooperation method provided by the above-mentioned embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0053] The electronic device provided by the embodiments of the present application is as follows: Figure 4As shown, the electronic device 400 includes a processor 402, a memory 401, and the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the method provided by the above embodiments.

[0054] Referring to Figure 4 , the electronic device further includes a bus 403 and a communication interface 404, and the processor 402, the communication interface 404 and the memory 401 are connected through the bus 403; the processor 402 is used to execute the executable modules stored in the memory 401, such as computer programs.

[0055] The memory 401 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication between the system network element and at least one other network element is realized through at least one communication interface 404 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0056] The bus 403 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0057] The memory 401 is used to store programs, and the processor 402 executes the programs after receiving execution instructions. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 402 or implemented by the processor 402.

[0058] The processor 402 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 402 or the instruction in the form of software. The processor 402 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 401, and the processor 402 reads the data in the memory 401, and combines the hardware to complete the steps of the above method.

[0059] Corresponding to the above-mentioned YTS-based vehicle-to-vehicle cooperation method, the embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the processor calls and runs the computer executable instructions, the computer executable instructions make the processor run the steps of the above-mentioned YTS-based vehicle-to-vehicle cooperation method.

[0060] The YTS-based vehicle-to-vehicle cooperation device provided by the embodiments of the present application can be specific hardware on the device or software or firmware installed on the device, etc. The device provided by the embodiments of the present application has the same implementation principle and generated technical effects as the foregoing method embodiments. For the sake of brevity, the part of the device embodiment not mentioned in the foregoing method embodiments can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can be referred to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0061] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely specific implementation manners of the present application, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electric, mechanical or other forms.

[0062] For another example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the apparatuses, methods and computer program products according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0063] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0064] In addition, each functional unit in the embodiments provided by the present application can be integrated into one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated into one unit.

[0065] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the human-computer collaboration method based on YTS for networked cars described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0066] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings, in addition, the terms "first", "second", "third" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0067] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, used to illustrate the technical solutions of the present application, and not to limit them, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: any person skilled in the art within the technical scope disclosed by the present application, they can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A human-machine collaboration method for connected vehicles based on YTS, characterized in that, The connected vehicle is equipped with onboard sensors; the method includes: The vehicle-mounted sensors acquire first environmental data about the surrounding environment of the connected vehicle. Based on the first environmental data and the control command triggered by the user corresponding to the connected vehicle, an operation to be executed is determined, and the operation to be executed has an execution time. The first environmental data is sent to the cloud, and the first environmental data is simulated on the cloud based on the YTS engine. When the execution time corresponding to the operation to be executed does not exceed the specified period, the operation to be executed is applied to the simulation environment of the YTS engine to obtain the execution result. The simulation environment is obtained by fusing the first environmental data and the second environmental data of other vehicles related to the location of the connected vehicle. Based on the execution result, the operation to be executed is optimized to obtain the optimized operation to be executed, and the optimized operation to be executed is sent to the connected vehicle; The execution time of the optimized operation to be executed is verified, and the optimized operation to be executed is executed at the corresponding verified time.

2. The human-machine collaboration method for connected vehicles based on the YTS engine according to claim 1, characterized in that, The process of building the simulation environment in the cloud includes: The cloud receives second environmental data from other vehicles within the geographical fence area where the connected vehicle is located; Based on the first environmental data and the second environmental data, a spatiotemporal alignment algorithm is used to fuse multi-source heterogeneous environmental data into a unified coordinate system, and a dynamic traffic situation model is established based on the fused environmental data in the unified coordinate system.

3. The human-machine collaboration method for connected vehicles based on YTS according to claim 2, characterized in that, The step of fusing multi-source heterogeneous environmental data into a unified coordinate system based on the first environmental data and the second environmental data using a spatiotemporal alignment algorithm, and establishing a dynamic traffic situation model based on the fused environmental data in the unified coordinate system, includes: The first environmental data and the second environmental data are mapped to a unified time and space coordinate system by affine transformation to obtain first environmental mapping data and second environmental mapping data; wherein, the first environmental data or the second environmental data includes at least three of the following: obstacle location data, road sign recognition results, meteorological monitoring data, traffic light status, and adjacent vehicle driving trajectories. Based on the first and second environment mapping data, a spatiotemporal alignment algorithm is used to fuse multi-source heterogeneous environment data into a unified coordinate system using the following formula: F ( p , t )= w 1 D 1( p , t )+ w 2 D 2( p , t );in, w 1 and w 2 is the weighting coefficient. D 1( p , t ) represents a specific location in the first environment mapping data. p In time t Data, D 1( p , t ) represents a specific location in the second environment mapping data. p In time t Data, F ( p , t ) represents the value obtained by fusing the first environment mapping data and the second environment mapping data; Based on the fused values, a dynamic traffic situation model is established using a fluid dynamics model. The dynamic traffic situation model includes at least one of traffic flow parameters, speed distribution parameters, and congestion parameters.

4. The human-machine collaboration method for connected vehicles based on YTS according to claim 2, characterized in that, The step of fusing multi-source heterogeneous environmental data into a unified coordinate system based on the first environmental data and the second environmental data using a spatiotemporal alignment algorithm includes: Based on the first environmental data and the second environmental data, the improved Kalman-Filter spatiotemporal fusion algorithm is used to fuse multi-source heterogeneous environmental data into a unified coordinate system using the following formula: k | k = k -1∣ k -1+ UK ; where Δ t The time difference between the first environmental data and the second environmental data. This is the identity matrix corresponding to the first environmental data and the second environmental data. UK The quantity of the control input corresponding to the control command. k -1∣ k -1 indicates the pre-fusion state estimation data from the previous time step. k | k This represents the fused state estimation data at the current moment.

5. The human-machine collaboration method for connected vehicles based on the YTS engine according to claim 1, characterized in that, The optimization of the operation to be executed based on the execution result to obtain the optimized operation to be executed includes: The operation parameters corresponding to the operation to be performed are iteratively optimized using a reinforcement learning algorithm to obtain the iterative optimization result. An optimized trajectory is generated based on the iterative optimization results and preset safety boundary constraints. The operational feasibility of the optimized trajectory is verified by digital twin method to obtain the operation verification results. Based on the operation verification results, the optimized operation to be executed is obtained.

6. The human-machine collaboration method for connected vehicles based on the YTS engine according to claim 1, characterized in that, The verification of the execution time of the optimized operation to be executed includes: A timestamp verification module is established, and the timestamp verification module is used to synchronize the cloud and vehicle system clocks via the NTP protocol to obtain the time synchronization result; Based on the time synchronization results, time window matching degree detection and remaining validity period calculation are performed before operation execution to obtain time detection results; If the time detection result meets the specified time verification index data, then the time verification result of the optimized operation to be executed is determined to be verified as passed.

7. The human-machine collaboration method for connected vehicles based on the YTS engine according to claim 1, characterized in that, The simulation environment in the cloud includes: a three-dimensional road network digital twin built based on the YTS engine, and integrates a V2X communication protocol stack simulator, a traffic flow micro-simulation module, and a vehicle dynamics model.

8. The human-machine collaboration method for connected vehicles based on the YTS engine according to claim 1, characterized in that, Sending the optimized operation to be executed to the connected vehicle includes: The operation instructions corresponding to the optimized operation to be executed are subjected to differential encryption to obtain the differential encryption result; An MQTT protocol transmission channel is established, and the differential encryption result is sent to the connected vehicle through the MQTT protocol transmission channel. The operation log corresponding to the operation instruction is recorded through blockchain notarization.

9. A human-machine collaboration device for connected vehicles based on YTS, characterized in that, The connected vehicle is equipped with onboard sensors, and the device includes: The acquisition module is used to acquire first environmental data around the connected vehicle through the on-board sensors; The determination module is used to determine the operation to be executed based on the first environmental data and the control command triggered by the user corresponding to the connected vehicle, wherein the operation to be executed has an execution time. The simulation module is used to send the first environmental data to the cloud, and to simulate the first environmental data on the cloud based on the YTS engine. When the execution time corresponding to the operation to be executed does not exceed the specified period, the operation to be executed is applied to the simulation environment of the YTS engine to obtain the execution result. The simulation environment is obtained by fusing the first environmental data and the second environmental data of other vehicles related to the location of the connected vehicle. An optimization module is used to optimize the operation to be executed based on the execution result, obtain the optimized operation to be executed, and send the optimized operation to be executed to the connected vehicle; The execution module is used to verify the execution time of the optimized operation to be executed, and to execute the optimized operation to be executed at the corresponding verified time.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 8.

Images