Remote assistance work order state machine control method and apparatus, electronic device, and medium

Abstract

The application discloses a remote assistance work order state machine control method and device, electronic equipment and medium, and belongs to the technical field of remote driving. The state machine at least includes an allocation state, a preparation state, a control state and a remote control completion state; the method comprises the following steps: in response to a remote assistance request of a vehicle, a remote assistance work order is generated, and a state machine corresponding to the remote assistance work order is set to the allocation state; in the allocation state, a work order allocation request is sent to a first cloud cabin, and in response to cloud cabin confirmation information returned by the first cloud cabin, the state is switched to the preparation state; in the preparation state, in response to successful control link establishment information of the vehicle and the first cloud cabin, the state is switched to the control state; in the control state, in response to control completion information sent by the first cloud cabin, the state is switched to the remote control completion state. The application can reduce the security risks caused by the state deviation among the cloud server, the vehicle and the cloud cabin, thereby improving the security of remote assistance.

Description

Technical Field

[0001] This application belongs to the field of remote driving technology, and in particular relates to a remote assistance work order state machine control method, device, electronic device and medium. Background Technology

[0002] Remote assistance is a driving aid mode that combines autonomous driving capabilities with remote human intervention. When an autonomous vehicle encounters a situation it cannot handle on its own, an operator can use remote commands or take over control to help the vehicle get out of trouble.

[0003] In related technologies, a dispatch server typically generates a work order based on the remote assistance request sent by the vehicle and assigns it to the cloud platform. The dispatch server then updates the work order status based on its own operation records. However, the actual status of the vehicle and the cloud platform may deviate from the status recorded by the dispatch server due to network latency, message loss, or node restarts. If the work order status is switched solely based on the dispatch server's unilateral operation records, dangerous situations can easily arise, such as the dispatch server believing the work order has ended when the vehicle is still waiting, or the cloud platform believing it has intervened in assistance when the vehicle has not yet opened its control channel. These situations seriously compromise the security of remote assistance. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a remote assistance work order state machine control method, device, electronic device, and medium to improve the security of remote assistance.

[0005] In a first aspect, this application provides a remote assistance work order state machine control method, wherein the state machine includes at least an allocation state, a preparation state, a control state, and a remote control completed state; the method includes: In response to a remote assistance request from a vehicle, a remote assistance work order is generated, and the state machine corresponding to the remote assistance work order is set to the allocation state. In the allocation state, a work order allocation request is sent to the first cloud cabin. In response to the cloud cabin confirmation information returned by the first cloud cabin, the process jumps to the preparation state. In the preparation state, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system jumps to the control state. In the controlled state, in response to the control completion information sent by the first cloud cabin, the system jumps to the remote control completion state.

[0006] The remote assistance work order state machine control method provided in this application designs the remote assistance work order state machine as a three-party state coordination protocol between the cloud server, vehicle, and cloud cabin. Each state corresponds to a specific consensus stage of the cloud server, vehicle, and cloud cabin, and the state transition is triggered by the explicit actions or verifiable events of each participating party. This can reduce the security risks caused by the three-party state deviation, thereby improving the security of remote assistance.

[0007] According to one embodiment of this application, the method further includes: In response to a current message sent by the vehicle, determine the vehicle identifier and semantic type corresponding to the current message; the semantic type includes at least one of assistance request, autonomous recovery, and abnormal event; Based on the vehicle identifier and semantic type corresponding to the current message, a search is performed in the historical message database. Based on the retrieved historical messages, the current message is deduplicated. The historical message database stores multiple sets of association relationships between vehicle identifiers, semantic types, and historical messages.

[0008] In this embodiment, by using mutually isolated deduplication logic for messages of different semantic types, for the current message sent by the vehicle, the semantic type corresponding to the current message is first determined, and then the current message is deduplicated based on the historical messages corresponding to the semantic type. This can reduce the situation where messages with the same response but different business meanings are misjudged as duplicate messages, leading to the triggering of erroneous state transitions.

[0009] According to one embodiment of this application, the method further includes: Before performing a state transition, obtain the state change request that triggers the state transition, and determine the current state of the state machine, the original state corresponding to the state change request, the target state corresponding to the state change request, the triggering party of the state change request, and each participating party corresponding to the state change request. If the original state matches the current state, the target state is a legitimate successor state of the current state, the triggering party has the authority to jump to the state, and all participating parties confirm all the conditions for the jump, then the jump proceeds to the target state.

[0010] In this embodiment, by verifying the state change request before performing the state transition, and then actually executing the state transition after the verification is successful, the situation of invalid state changes, unauthorized triggering parties, and asynchronous states of participating parties leading to erroneous transitions can be reduced, thereby improving the accuracy of state transitions.

[0011] According to one embodiment of this application, the method further includes: After navigating to the target state, a state change record is generated based on the current time, the state change request, the current state, the target state, the triggering party, and the participating party. The status change record is stored in the log corresponding to the remote assistance work order.

[0012] In this embodiment, by generating a status change record and storing it in the log corresponding to the remote assistance work order after the status transition is completed, the status changes during the remote assistance process can be fully recorded for subsequent safety audits and incident reviews.

[0013] According to one embodiment of this application, the allocation status includes a queuing waiting stage, a work order issuance stage, and a resource binding stage; The step of generating a remote assistance work order and setting the state machine corresponding to the remote assistance work order to the allocation state includes: generating the remote assistance work order and setting the state machine corresponding to the remote assistance work order to the queuing waiting stage; In the allocation state, a work order allocation request is sent to the first cloud cabin. In response to the cloud cabin confirmation information returned by the first cloud cabin, the process jumps to the preparation state, including: During the queuing and waiting phase, a work order allocation request is sent to the first cloud cabin, and the process jumps to the work order issuance phase. During the work order issuance stage, in response to the cloud cabin confirmation information sent by the first cloud cabin, the process jumps to the resource binding stage; During the resource binding phase, resource binding records are created for the vehicle, the first cloud cabin, and the remote assistance work order. If the resource binding record is successfully created, the process jumps to the preparation state.

[0014] In this embodiment, after receiving the cloud cabin confirmation information sent by the first cloud cabin, a resource binding record is created between the vehicle, the first cloud cabin, and the remote assistance work order. If the resource binding record is successfully created, the system jumps to the preparation stage. This can reduce concurrent conflicts such as multiple cloud cabins competing for the same work order, the same cloud cabin serving multiple work orders at the same time, or the same vehicle being occupied by multiple work orders, thereby improving the accuracy of status transition.

[0015] According to one embodiment of this application, the method further includes, in at least one of the following cases, terminating in the queuing phase: During the work order issuance phase, the first cloud cabin returns a cloud cabin rejection message; During the resource binding phase, the creation of the resource binding record failed. In the preparation state, the first cloud cabin sends a response error message.

[0016] In this embodiment, by jumping back to the queuing stage to re-trigger allocation in abnormal situations such as cloud cabin refusing to accept orders, resource binding failure, or abnormal cloud cabin response, the situation where remote assistance work orders remain in the intermediate state for a long time due to abnormalities can be reduced, and the accuracy of state transition can be improved.

[0017] According to one embodiment of this application, the preparation state includes an operator confirmation stage, an observation link establishment stage, and a control link establishment stage; The step of jumping to the preparation state in response to the cloud cabin confirmation information returned by the first cloud cabin includes: jumping to the operator confirmation stage in response to the cloud cabin confirmation information returned by the first cloud cabin; In the preparation state, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state, including: During the operator confirmation phase, in response to the operator confirmation information sent by the operator through the first cloud cabin, the process jumps to the observation link establishment phase. During the observation link establishment phase, in response to the successful establishment of the observation link between the vehicle and the first cloud cabin, the process jumps to the control link establishment phase. During the control link establishment phase, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state.

[0018] In this embodiment, by sequentially receiving operator confirmation information sent by the operator through the first cloud cabin and information indicating successful establishment of the observation link between the vehicle and the first cloud cabin, and then responding to information indicating successful establishment of the control link between the vehicle and the first cloud cabin, the system jumps to the control state. This reduces the risk of starting remote control without operator confirmation or observation conditions, thereby improving the safety of remote assistance.

[0019] According to one embodiment of this application, the state machine further includes a degradation assignment state, a degradation preparation state, and a degradation control state; the method further includes: In the preparation state, in response to the control downgrade information sent by the operator through the first cloud cabin, the system jumps to the downgrade allocation state. In the downgrade allocation state, the work order allocation request is sent to the second cloud cabin. In response to the confirmation information returned by the second cloud cabin, the process jumps to the downgrade preparation state. The first cloud cabin corresponds to the command vehicle control mode, and the second cloud cabin corresponds to the driving vehicle control mode. In the state of preparation for degradation, in response to the successful establishment of the control link between the vehicle and the second cloud cabin, the system jumps to the state of control for degradation. In the degraded control state, in response to the control completion information sent by the second cloud cabin, the system jumps to the remote control completion state.

[0020] In this embodiment, by setting up two cloud cabins corresponding to different vehicle control modes, when the operator sends control downgrade information through the first cloud cabin corresponding to the vehicle control mode, the downgrade process is initiated, and the second cloud cabin corresponding to the driving control mode provides remote assistance to the vehicle, which can improve the accuracy of remote assistance.

[0021] According to one embodiment of this application, the state machine further includes an autonomous recovery state; the method further includes: In at least one of the allocation state and the preparation state, in response to autonomous recovery information sent by the vehicle, the system jumps to the autonomous recovery state.

[0022] In this embodiment, by responding to the autonomous recovery information sent by the vehicle, the system jumps to the autonomous recovery state, which can reduce unnecessary remote takeover operations.

[0023] According to one embodiment of this application, the state machine further includes an abnormal termination state; the method further includes: In at least one of the allocation state, the preparation state, and the control state, in response to an abnormal interruption event, the process jumps to the abnormal termination state. The abnormal interruption event includes at least one of the following: the vehicle sending external interference information, the vehicle sending control link interruption information, and the first cloud cabin sending control link interruption information.

[0024] In this embodiment, by maintaining multiple anomaly detection mechanisms and jumping to an abnormal termination state when an abnormal interruption event is detected, it can be distinguished from the normally completed remote control completion state, which facilitates subsequent on-site handling or manual review, while reducing the situation where remote assistance work orders remain in the intermediate state for a long time due to anomalies, and improving the accuracy of state transition.

[0025] According to one embodiment of this application, the method further includes: In the controlled state, in response to the abnormal interruption event, a control link disconnection command is sent to the first cloud cabin to cause the first cloud cabin to release the control link between the first cloud cabin and the vehicle.

[0026] In this embodiment, by causing the first cloud cabin to release the control link between the first cloud cabin and the vehicle when an abnormal interruption event occurs in the controlled state, the occurrence of the vehicle executing erroneous remote control commands under abnormal circumstances can be reduced, thereby improving the safety of vehicle operation.

[0027] Secondly, this application provides a remote assistance work order state machine control device, comprising: The generation module is used to generate a remote assistance work order in response to a remote assistance request from a vehicle, and set the state machine corresponding to the remote assistance work order to the allocation state. The first jump module is used to send a work order allocation request to the first cloud cabin in the allocation state, and jump to the preparation state in response to the cloud cabin confirmation information returned by the first cloud cabin. The second jump module is used to jump to the control state in response to the successful establishment of the control link between the vehicle and the first cloud cabin in the preparation state. The third jump module is used to jump to the remote control completion state in response to the control completion information sent by the first cloud cabin in the control state.

[0028] According to the remote assistance work order state machine control device of this application, by designing the remote assistance work order state machine as a three-party state coordination protocol between the cloud server, vehicle and cloud cabin, each state corresponds to a specific consensus stage of the cloud server, vehicle and cloud cabin, and the state transition is triggered by the explicit actions or verifiable events of each participant, the security risks caused by the three-party state deviation can be reduced, thereby improving the security of remote assistance.

[0029] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the remote assistance work order state machine control method as described in the first aspect above.

[0030] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the remote assistance work order state machine control method as described in the first aspect above.

[0031] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the remote assistance work order state machine control method as described in the first aspect above.

[0032] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the remote assistance work order state machine control method as described in the first aspect above.

[0033] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: According to the remote assistance work order state machine control method of this application, by designing the remote assistance work order state machine as a three-party state coordination protocol between the cloud server, vehicle and cloud cabin, each state corresponds to a specific consensus stage of the cloud server, vehicle and cloud cabin, and the state transition is triggered by the explicit actions or verifiable events of each participant, the security risks caused by the three-party state deviation can be reduced, thereby improving the security of remote assistance.

[0034] In some embodiments, by employing mutually isolated deduplication logic for messages of different semantic types, for the current message sent by the vehicle, the semantic type corresponding to the current message is first determined, and then the current message is deduplicated based on the historical messages corresponding to that semantic type. This can reduce the situation where messages with the same response but different business meanings are misjudged as duplicate messages, leading to the triggering of erroneous state transitions.

[0035] In some embodiments, by validating the state change request before performing a state transition, and then actually performing the state transition after the validation is successful, the situation of erroneous transitions caused by invalid state changes, unauthorized triggering parties, or asynchronous states of participating parties can be reduced, thereby improving the accuracy of state transitions.

[0036] In some embodiments, by generating a status change record and storing it in the log corresponding to the remote assistance work order after the status transition is completed, the status changes during the remote assistance process can be fully recorded for subsequent security audits and incident reviews.

[0037] In some embodiments, by creating a resource binding record between the vehicle, the first cloud cabin, and the remote assistance work order after receiving the cloud cabin confirmation information sent by the first cloud cabin, and jumping to the preparation stage when the resource binding record is successfully created, the concurrent conflicts of multiple cloud cabins competing for the same work order, the same cloud cabin serving multiple work orders at the same time, or the same vehicle being occupied by multiple work orders can be reduced, thereby improving the accuracy of status transition.

[0038] In some embodiments, by jumping back to the queuing waiting stage to re-trigger allocation in abnormal situations such as cloud cabin refusing to accept orders, resource binding failure, or abnormal cloud cabin response, the situation where remote assistance work orders remain in the intermediate state for a long time due to abnormalities can be reduced, and the accuracy of state transition can be improved.

[0039] In some embodiments, by sequentially receiving operator confirmation information sent by the operator through the first cloud cabin and information indicating successful establishment of the observation link between the vehicle and the first cloud cabin, and then responding to information indicating successful establishment of the control link between the vehicle and the first cloud cabin, the system jumps to the control state. This reduces the risk of starting remote control without operator confirmation or observation conditions, thereby improving the safety of remote assistance.

[0040] In some embodiments, by setting up two cloud cabins corresponding to different vehicle control modes, when the operator sends control downgrade information through the first cloud cabin corresponding to the command vehicle control mode, the downgrade process is initiated, and the second cloud cabin corresponding to the driving vehicle control mode provides remote assistance to the vehicle, which can improve the accuracy of remote assistance.

[0041] In some embodiments, by switching to an autonomous recovery state in response to autonomous recovery information sent by the vehicle, unnecessary remote takeover operations can be reduced.

[0042] In some embodiments, by maintaining multiple anomaly detection mechanisms and jumping to an abnormal termination state when an abnormal interruption event is detected, it is possible to distinguish it from the normally completed remote control completion state, which facilitates subsequent on-site handling or manual review, while reducing the situation where remote assistance work orders remain in the intermediate state for a long time due to anomalies, and improving the accuracy of state transition.

[0043] In some embodiments, by causing the first cloud cabin to release the control link between the first cloud cabin and the vehicle when an abnormal interruption event occurs in the controlled state, the occurrence of the vehicle executing erroneous remote control commands under abnormal circumstances can be reduced, thereby improving the safety of vehicle operation.

[0044] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the remote assistance work order state machine control method provided in the embodiments of this application; Figure 2 These are examples of scenarios for state transitions in the state machine provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of the remote assistance work order state machine control device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0048] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0049] The remote assistance work order state machine control method, device, electronic device and medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0050] The remote assistance work order state machine control method provided in this application embodiment can be executed by a scheduling server or a functional module or entity within the scheduling server capable of implementing the remote assistance work order state machine control method. The scheduling server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0051] The following uses a scheduling server as the execution subject as an example to illustrate the remote assistance work order state machine control method provided in this application embodiment.

[0052] In this embodiment of the application, a state machine corresponding to the remote assistance work order is constructed, which includes at least the allocation state, the preparation state, the control state, and the remote control completed state.

[0053] The "Assigning" status refers to the status after a remote assistance work order is created. In the "Assigning" status, the scheduling server assigns the remote assistance work order to the first cloud module and waits for confirmation of receipt from the first cloud module.

[0054] The "Ready" status indicates that the first cloud cabin has confirmed receipt of the remote assistance work order. In this status, the first cloud cabin and the vehicle negotiate and establish a control link to prepare for remote assistance.

[0055] The "Controlled" state refers to the state after the control link between the first cloud cabin and the vehicle has been successfully established. In the "Controlled" state, the first cloud cabin takes over control of the vehicle. The operator inputs remote control commands through the first cloud cabin, which transmits the remote control commands to the vehicle through the established control link. The vehicle receives and executes the remote control commands.

[0056] The remote control completion status indicates that the first cloud cabin has confirmed the completion of remote control. In this status, the remote assistance task corresponding to the remote assistance request has been successfully completed, and the scheduling server performs tasks such as work order archiving and resource release.

[0057] like Figure 1 As shown, the remote assistance work order state machine control method includes steps 110, 120, 130 and 140.

[0058] Step 110: In response to the vehicle's remote assistance request, generate a remote assistance work order and set the state machine corresponding to the remote assistance work order to the allocation state.

[0059] In this embodiment, a remote assistance request refers to a request reported by a vehicle to a dispatch server when encountering complex road conditions, autonomous driving system malfunctions, or other scenarios that it cannot handle autonomously. The remote assistance request may carry information such as vehicle identification, vehicle type, vehicle location, and type of assistance requested. The remote assistance request can be initiated by the vehicle's autonomous driving system in cases of path planning failure or response timeout, or it can be initiated manually by the driver, passengers, or safety personnel in the vehicle, or it can be initiated by remote monitoring personnel or remote monitoring systems based on vehicle images or vehicle alarm information. Of course, remote assistance requests can also be initiated through other means, and this embodiment does not limit this approach.

[0060] In response to a vehicle's remote assistance request, the dispatch server generates a remote assistance work order. A remote assistance work order is structured data corresponding to a remote assistance request, used to record, track, and assign remote assistance tasks. The dispatch server can parse the remote assistance request to obtain information such as vehicle identification, vehicle type, vehicle location, and requested assistance type. Based on the parsed information and a preset remote assistance work order template, the server creates the corresponding remote assistance work order for that request.

[0061] After creating a remote assistance work order, you can create a state machine instance corresponding to the remote assistance work order and set the state machine to the allocation state.

[0062] Step 120: In the allocation state, send a work order allocation request to the first cloud cabin. In response to the cloud cabin confirmation information returned by the first cloud cabin, jump to the preparation state.

[0063] In this embodiment of the application, when the allocation is in progress, the scheduling server allocates the remote assistance work order to the first cloud cabin and waits for the first cloud cabin to confirm receipt.

[0064] In the allocation phase, the scheduling server sends a work order allocation request to the first cloud module. This work order allocation request represents a request for the first cloud module to receive a remote assistance work order, and may include information such as the remote assistance work order and its validity period.

[0065] After receiving a work order allocation request, the first cloud cabin can determine whether to accept the remote assistance work order based on its current number of pending work orders and its health status. If it determines to accept the remote assistance work order, it can return a cloud cabin confirmation message to the scheduling server in response to the work order allocation request.

[0066] After the scheduling server receives the cloud cabin confirmation information returned by the first cloud cabin, it can determine that the first cloud cabin has confirmed that it has received the current remote assistance work order, and then jumps to the preparation state.

[0067] Step 130: In the preparation state, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system jumps to the control state.

[0068] In this embodiment of the application, during the preparation state, the first cloud cabin and the vehicle negotiate to establish a control link in preparation for remote assistance. The control link refers to the end-to-end communication channel between the first cloud cabin and the vehicle, used to reliably transmit control commands issued by the first cloud cabin to the vehicle, and simultaneously transmit vehicle status, location, telemetry data, etc., back to the first cloud cabin.

[0069] Remote assistance depends on the establishment of a control link. Therefore, in the preparation state, the dispatch server needs to switch to the control state after receiving the information that the control link between the vehicle and the first cloud cabin has been successfully established.

[0070] The successful control link establishment information indicates that a remote control link has been successfully established between the vehicle and the first cloud module. This information can include confirmation messages sent by the vehicle to the dispatch server after confirming the successful establishment of the control link, as well as confirmation messages sent by the first cloud module to the dispatch server after confirming the successful establishment of the control link. Upon receiving confirmation messages from both the vehicle and the first cloud module, the system can determine that the control link between the vehicle and the first cloud module has been successfully established and proceed to the control state.

[0071] Step 140: In the control state, in response to the control completion information sent by the first cloud cabin, jump to the remote control completion state.

[0072] In this embodiment of the application, under control, the first cloud cabin takes over the control of the vehicle. The operator inputs remote control commands through the first cloud cabin, and the first cloud cabin transmits the remote control commands to the vehicle through the established control link. The vehicle receives and executes the remote control commands.

[0073] Once the operator confirms that the vehicle has left the complex environment and can continue operating in autonomous driving mode, or has been parked in a safe location and can be shut down or manually taken over driving, control can be confirmed through the first cloud cabin. After receiving the confirmation information input by the operator, the first cloud cabin can send control completion information to the dispatch server.

[0074] Upon receiving control completion information from the first cloud cabin, the dispatch server can determine that the remote control task has been successfully completed and transition to the remote control completion state. Alternatively, after receiving control completion information from the first cloud cabin, the dispatch server can further confirm whether the vehicle has confirmed remote control completion, and transition to the remote control completion state only if both the first cloud cabin and the vehicle have sent control completion information.

[0075] When remote control is complete, the remote assistance task corresponding to the remote assistance request has been completed normally. The scheduling server can close the remote assistance work order after completing the work order archiving and resource release.

[0076] The remote assistance work order state machine control method provided in this application design the remote assistance work order state machine as a three-party state coordination protocol between the cloud server, vehicle, and cloud cabin. Each state corresponds to a specific consensus stage of the cloud server, vehicle, and cloud cabin, and the state transition is triggered by the explicit actions or verifiable events of each participant. This can reduce the security risks caused by the three-party state deviation, thereby improving the security of remote assistance.

[0077] In some embodiments, the method further includes: In response to the current message sent by the vehicle, determine the vehicle identifier and semantic type corresponding to the current message; the semantic type includes at least one of assistance request, autonomous recovery, and abnormal event; Based on the vehicle identifier and semantic type corresponding to the current message, the system retrieves information from the historical message database. Based on the retrieved historical messages, the current message is deduplicated. The historical message database stores multiple sets of associations between vehicle identifiers, semantic types, and historical messages.

[0078] In situations involving network fluctuations or multiple retransmissions triggered by vehicles, the dispatch server may receive identical messages from the same vehicle within a short period. If a corresponding work order creation and status transition are executed for each received message, it could lead to duplicate work order creation, status transition failures, or errors. Therefore, the dispatch server needs to perform deduplication on received messages, i.e., it needs to determine whether the currently received message is a duplicate of a historical message; if a duplicate exists, the current message is ignored.

[0079] Furthermore, besides the possibility of mistakenly sending duplicate messages under abnormal circumstances, the same vehicle may also send multiple messages with different business meanings consecutively within a short period. Simply deduplicating messages based on their corresponding vehicle identifiers could lead to messages from the same vehicle but with different business meanings being misjudged as duplicates, resulting in work orders that should be created being incorrectly ignored, or work orders that should be terminated continuing to wait. Therefore, in this embodiment, deduplication logic that isolates messages of different semantic types is used.

[0080] After receiving the current message sent by the vehicle, the dispatch server first determines the vehicle identifier and semantic type corresponding to the current message.

[0081] Among them, vehicle identification is identification information representing the vehicle's identity, and semantic type represents the business meaning of the current message. In the embodiments of this application, semantic type includes at least one of assistance request, autonomous recovery, and abnormal event. Specifically, a message with semantic type of assistance request indicates that the vehicle encounters a scenario that it cannot handle autonomously during operation and requires cloud cabin intervention, and can be a remote assistance request, etc.; a message with semantic type of autonomous recovery indicates that the vehicle has escaped the abnormal scenario or regained its autonomous driving capability, and can be autonomous recovery information, etc.; a message with semantic type of abnormal event indicates that the vehicle encounters abnormal situations such as control link interruption, manual intervention, accident, hardware failure, etc., and can be alarm information, external interference information, control link interruption information, etc. Of course, each semantic type can also correspond to other forms of messages, and messages can also be of other semantic types, which are not limited in this embodiment of the application.

[0082] Vehicle identifiers can be pre-configured when vehicles leave the factory, or assigned by the management or dispatch platform when the vehicle is powered on. The correspondence between message content and semantic type can be pre-defined and stored locally on the vehicle. When a vehicle sends a message, it can determine the semantic type based on the message content and write the vehicle identifier and semantic type into the message's header field. The dispatch server can directly parse the vehicle identifier and semantic type from the current message's header field.

[0083] Of course, vehicle identifiers can be configured and message semantic types can be determined by other methods, and the vehicle identifier and semantic type corresponding to the current message can also be determined by other methods. This application embodiment does not limit this.

[0084] The scheduling server determines the vehicle identifier and semantic type corresponding to the current message, and then searches the historical message database based on the vehicle identifier and semantic type corresponding to the current message.

[0085] The historical message database stores multiple sets of associations between vehicle identifiers, semantic types, and historical messages. This allows the scheduling server to associate a message with its corresponding vehicle identifier and semantic type and store it in the historical message database after receiving the message, thus constructing the historical message database.

[0086] The scheduling server then searches the historical message database based on the vehicle identifier and semantic type corresponding to the current message. It can retrieve historical messages with the same vehicle identifier and semantic type as the current message. Therefore, by deduplicating the current message based on the retrieved historical messages, the possibility of messages with different business meanings being mistakenly identified as duplicate messages can be reduced.

[0087] Of course, in addition to determining the deduplication logic based on vehicle identification and semantic type, other information, such as message sending time, can be introduced during deduplication. This application embodiment does not limit this.

[0088] In this embodiment, by using mutually isolated deduplication logic for messages of different semantic types, for the current message sent by the vehicle, the semantic type corresponding to the current message is first determined, and then the current message is deduplicated based on the historical messages corresponding to the semantic type. This can reduce the situation where messages with the same response but different business meanings are misjudged as duplicate messages, leading to the triggering of erroneous state transitions.

[0089] In some embodiments, the method further includes: Before performing a state transition, obtain the state change request that triggers the state transition, and determine the current state of the state machine, the original state corresponding to the state change request, the target state corresponding to the state change request, the triggering party of the state change request, and each participant corresponding to the state change request. Jump to the target state if the original state matches the current state, the target state is a valid successor state of the current state, the triggering party has the authority to jump to the state, and all participating parties confirm all the conditions for the jump.

[0090] In this embodiment, a state change request triggers a state transition in the state machine. The state change request represents the desired change in the state machine's state and can be a request generated by the scheduling server based on received messages. For example, if the scheduling server receives information indicating a successful establishment of the control link between the vehicle and the first cloud cabin, it can generate a state change request indicating a desired change from the "ready" state to the "controlled" state.

[0091] Due to factors such as network latency, message loss, node restarts, and external attacks, state change requests may suffer from issues such as invalid state changes or mismatches between the changed content and the current state of the state machine. Therefore, in this embodiment, the state change request that triggers the state transition is validated before the transition is performed.

[0092] When a state change request is received, the current state of the state machine, the original state corresponding to the state change request, the target state corresponding to the state change request, the triggering party of the state change request, and each participant corresponding to the state change request are first determined.

[0093] In this context, the current state of the state machine refers to the state the state machine is in at the moment the state change request is received; the original state corresponding to the state change request refers to the original state of the state machine in the state change request; the target state corresponding to the state change request refers to the state to which the state change request expects the state machine to change. For example, if the state change request represents the expectation to change the state machine from "preparing state" to "controlling state", then the original state is "preparing state" and the target state is "controlling state"; the triggering party of the state change request refers to the business entity from which the message triggering this state transition comes, which can be the vehicle, cloud cabin, scheduling server itself, etc. For example, if the scheduling server generates a state change request in response to the control link establishment success information sent by the vehicle, then the triggering party corresponding to this state change request is the vehicle; the participating party corresponding to the state change request refers to the business entity involved in this state transition event, which can be the vehicle, cloud cabin, scheduling server itself, etc.

[0094] In this embodiment, the matching of the original state and the current state is verified. The original state and the current state can be compared, and if they are the same, the original state and the current state are determined to match. For example, if the original state corresponding to a state change request is "preparing state," and the current state of the state machine is also "preparing state," then the original state and the current state are determined to match. If the current state of the state machine is a state other than "preparing state," such as "allocation state," then the original state and the current state are determined not to match.

[0095] In this embodiment, it is verified whether the target state is a valid successor state to the current state. This can be determined based on predefined state transition rules. For example, if the original state is "Preparing State" and the target state is "Controlling State," and the state transition rules define a transition from "Preparing State" to "Controlling State," then the target state can be determined to be a valid successor state to the original state. However, if the original state is "Assigning State" and the target state is "Controlling State," and the state transition rules do not define a transition from "Preparing State" to "Controlling State," then the target state cannot be determined to be a valid successor state to the original state.

[0096] In this embodiment, it is verified whether the triggering party has the permission to transition states. Permission verification rules corresponding to each transition rule can be pre-defined in the state transition rules, and then the permission verification rules can be used to determine whether the triggering party has the permission to transition states. For example, a vehicle does not have the permission to change its state from "assigning" to "preparing," and vehicle 1 does not have the permission to change the state of the remote assistance work order corresponding to vehicle 2.

[0097] In this embodiment, it is verified whether all participating parties have confirmed the jump. The state jump event may involve multiple business entities. For example, the participating parties involved in the jump from "preparing state" to "controlling state" include the vehicle and the first cloud cabin. If the scheduling server only receives the confirmation information sent by the first cloud cabin after confirming the successful establishment of the control link, but does not receive the confirmation information sent by the vehicle after confirming the successful establishment of the control link, it can be determined that there are participating parties who have not confirmed the jump.

[0098] If the original state matches the current state, the target state is a valid successor state of the current state, the triggering party has the authority to jump to the state, and all participating parties confirm all the conditions in the jump, the verification can be deemed successful, and the state jump operation can be started, causing the state machine to jump to the target state corresponding to the state change request.

[0099] Of course, other methods can be used for various verifications; other verification rules can also be set to verify other information associated with the status change request. For example, it can be verified whether the status change request involves unreleased vehicles, cloud cabins, or control link resources, and whether the content of the event that triggered the status change request matches the target status, etc. This application embodiment does not limit these aspects.

[0100] In this embodiment, by verifying the state change request before performing the state transition, and then actually executing the state transition after the verification is successful, the situation of invalid state changes, unauthorized triggering parties, and asynchronous states of participating parties leading to erroneous transitions can be reduced, thereby improving the accuracy of state transitions.

[0101] In some embodiments, the method further includes: After navigating to the target state, a state change record is generated based on the current time, state change request, current state, target state, triggering party, and participating parties. Status change records are stored in the log corresponding to the remote assistance work order.

[0102] To facilitate subsequent security audits and incident reviews, in this embodiment of the application, a status change record is written to the log corresponding to the remote assistance work order after each status transition.

[0103] The current state, target state, triggering party, and participating parties determined when verifying a state change request can be reused, and a state change record containing the current time, state change request, current state, target state, triggering party, and participating parties can be generated according to a preset record format. Of course, other information can also be recorded in the state change record, such as the type and content of the abnormal event when a state transition is triggered by an abnormal event, etc., but this application embodiment does not limit this.

[0104] After generating a status change record, store it in the log corresponding to the remote assistance work order. An append-only approach can be used, storing the currently generated status change records sequentially at the end of the log, ensuring that the records are arranged chronologically and preventing historical status change records from being overwritten or deleted.

[0105] Therefore, the logs corresponding to remote assistance work orders can record the records corresponding to each status change during the lifecycle of the remote assistance work order, thus forming a complete event tracing chain. The system can be queried by work order, vehicle, cloud cabin, time range, etc., for accident review, responsibility determination, and safety auditing.

[0106] In this embodiment, by generating a status change record and storing it in the log corresponding to the remote assistance work order after the status transition is completed, the status changes during the remote assistance process can be fully recorded for subsequent safety audits and incident reviews.

[0107] In some embodiments, the allocation status includes a queuing waiting stage, a work order issuance stage, and a resource binding stage; Generate a remote assistance work order and set the state machine corresponding to the remote assistance work order to the allocation state, including: generating a remote assistance work order and setting the state machine corresponding to the remote assistance work order to the queuing waiting stage; In the allocation phase, a work order allocation request is sent to the first cloud module. Upon receiving confirmation from the first cloud module, the process transitions to the preparation phase, including: During the queuing and waiting phase, a work order allocation request is sent to the first cloud cabin, and the process jumps to the work order issuance phase. During the work order issuance phase, in response to the cloud cabin confirmation information sent by the first cloud cabin, the process jumps to the resource binding phase. During the resource binding phase, resource binding records are created for the vehicle, the first cloud cabin, and the remote assistance work order. Once the resource binding records are successfully created, the process will jump to the preparation state.

[0108] In this application embodiment, the allocation status includes the queuing waiting stage, the work order issuance stage, and the resource binding stage.

[0109] When a large number of remote assistance requests are concurrently received, there may not be a primary cloud container capable of receiving the current remote assistance work order when generating the corresponding work order. If work order allocation requests are continued to be sent to the primary cloud container in this situation, it may lead to remote assistance work orders with different creation times competing for cloud container resources, resulting in some remote assistance work orders being unable to be processed.

[0110] Therefore, in this embodiment, after the scheduling server generates a remote assistance work order, it first sets the state machine corresponding to the remote assistance work order to a queuing waiting stage. The queuing waiting stage is the stage after the remote assistance work order is created. During the queuing waiting stage, the remote assistance work order waits for the scheduling server to allocate it to the first cloud container. The scheduling server can store the current remote assistance work order in a scheduling queue and allocate each remote assistance work order to an available first cloud container in the order they entered the scheduling queue. After sending a work order allocation request corresponding to the current remote assistance work order to the first cloud container, the scheduling server causes the state machine corresponding to the current remote assistance work order to jump to the work order issuance stage.

[0111] The work order issuance phase occurs after the scheduling server sends a work order allocation request to the first cloud container. During this phase, the scheduling server has already sent the work order allocation request to the first cloud container and is waiting for the first cloud container to return confirmation information indicating that it has received the remote assistance work order.

[0112] In situations with a large number of concurrent remote assistance requests, concurrency conflicts may arise, such as multiple cloud cabins vying for the same work order, the same cloud cabin serving multiple work orders simultaneously, or the same vehicle being occupied by multiple work orders. Therefore, in this embodiment, after receiving the cloud cabin confirmation information returned by the first cloud cabin, the system jumps to the resource binding stage instead of directly entering the confirmation stage. The resource binding stage occurs after the first cloud cabin returns the cloud cabin confirmation information. During the resource binding stage, the scheduling server creates resource binding records corresponding to the vehicle, the first cloud cabin, and the remote assistance work order.

[0113] In this embodiment, the resource binding record represents the atomic binding relationship between the vehicle, the first cloud cabin, and the remote assistance work order. This ensures that the same vehicle participates in only one active remote assistance work order at any given time, the same cloud cabin serves only one active remote assistance work order at any given time, and the same remote assistance work order can only be effectively bound to one specific vehicle and one specific cloud cabin at any given time. Resource binding records corresponding to the vehicle, the first cloud cabin, and the remote assistance work order can be created through transactions, locks, uniqueness constraints, or other equivalent concurrency control methods; this embodiment does not limit this approach.

[0114] If any of the vehicle, first cloud cabin, or remote assistance work orders has an existing or in-process resource binding record, indicating a concurrent contention situation, then the resource binding record creation can be considered a failure. In the event of a resource binding record creation failure, the scheduling server can select the only valid resource binding record based on creation time and other factors, and initiate other resource binding processes into a release, retry, or reallocation process.

[0115] If the resource binding record is successfully created, the state machine will transition to the ready state.

[0116] In this embodiment, after receiving the cloud cabin confirmation information sent by the first cloud cabin, a resource binding record is created between the vehicle, the first cloud cabin, and the remote assistance work order. If the resource binding record is successfully created, the system jumps to the preparation stage. This can reduce concurrent conflicts such as multiple cloud cabins competing for the same work order, the same cloud cabin serving multiple work orders at the same time, or the same vehicle being occupied by multiple work orders, thereby improving the accuracy of status transition.

[0117] In some embodiments, the method further includes terminating in a queuing phase in at least one of the following cases: During the work order issuance phase, the first cloud cabin returned a cloud cabin rejection message; During the resource binding phase, the creation of the resource binding record failed. While in the preparation state, the first cloud cabin sent a response error message.

[0118] Before actual remote control begins, during the work order issuance, resource binding, or preparation phases, abnormal situations may occur such as cloud service provider refusing to accept the order, failure to confirm in a timely manner, or concurrent contention. If the conditions for status transition are not met at this point, the process cannot proceed to the next stage and trigger the corresponding processing flow, causing the remote assistance work order to remain in the intermediate state for an extended period.

[0119] Therefore, in some abnormal situations, the state machine can jump back to the queuing waiting stage to re-trigger the queuing and allocation process.

[0120] During the work order issuance phase, the first cloud cabin may return a cloud cabin rejection message to the scheduling server because it is unable to receive new remote assistance work orders due to other ongoing work orders. When the scheduling server receives the cloud cabin rejection message returned by the first cloud cabin, it adds the current remote assistance work order back to the scheduling queue and causes the state machine to jump back to the queuing waiting phase.

[0121] During the resource binding phase, the first cloud cabin to which the current remote assistance work order is assigned may simultaneously receive work order assignment requests from other corresponding remote assistance work orders. If the scheduling server decides to terminate the resource binding process corresponding to the current remote assistance work order and the resource binding record creation fails, the current remote assistance work order will be added back to the scheduling queue, and the state machine will jump back to the queuing waiting phase.

[0122] In the preparation state, the first cloud cabin may return an abnormal response message due to reasons such as the operator's failure to confirm in time or hardware failure of the first cloud cabin. At this time, the current first cloud cabin cannot perform remote assistance tasks. When the scheduling server receives the abnormal response message sent by the first cloud cabin, it can release the resource binding record corresponding to the vehicle, the current first cloud cabin and the current remote assistance work order, add the current remote assistance work order back to the scheduling queue, and cause the state machine to jump back to the queuing waiting stage.

[0123] Of course, other triggering conditions for jumping to the queuing waiting stage can also be set, but this application embodiment does not limit this.

[0124] In this embodiment, by jumping back to the queuing stage to re-trigger allocation in abnormal situations such as cloud cabin refusing to accept orders, resource binding failure, or abnormal cloud cabin response, the situation where remote assistance work orders remain in the intermediate state for a long time due to abnormalities can be reduced, and the accuracy of state transition can be improved.

[0125] In some embodiments, the preparation state includes an operator confirmation phase, an observation link establishment phase, and a control link establishment phase; In response to the cloud cabin confirmation information returned by the first cloud cabin, the system jumps to the preparation state, including: in response to the cloud cabin confirmation information returned by the first cloud cabin, the system jumps to the operator confirmation stage; In the preparation state, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state, including: During the operator confirmation phase, in response to the operator confirmation information sent by the operator through the first cloud cabin, the process jumps to the observation link establishment phase. During the observation link establishment phase, in response to the successful establishment of the observation link between the vehicle and the first cloud cabin, the system jumps to the control link establishment phase. During the control link establishment phase, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state.

[0126] During remote assistance, the operator needs to observe the vehicle's environment through video transmitted from the vehicle's onboard camera before issuing remote control commands. If a control link is established and the first cloud cabin takes over vehicle control before the operator is present, has completed takeover preparations, or has the necessary observation capabilities, it could lead to higher security risks.

[0127] Therefore, in this embodiment, the preparation state includes an operator confirmation stage, an observation link establishment stage, and a control link establishment stage. The operator confirmation stage occurs after the first cloud cabin returns confirmation information. When the scheduling server receives the confirmation information from the first cloud cabin, it first causes the state machine to jump to the operator confirmation stage.

[0128] During the operator confirmation phase, the first cloud cabin can display relevant information about the current remote assistance work order to the operator through devices such as monitors and speakers. The operator can input confirmation information using input devices integrated into the first cloud cabin, such as mouse, keyboard, steering wheel, and pedals. The first cloud cabin can forward the operator confirmation information to the scheduling server. When the scheduling server receives the operator confirmation information, it can determine that the operator in the first cloud cabin is ready and instruct the state machine to jump to the observation link establishment phase.

[0129] The observation link establishment phase occurs after the operator confirms receipt of the remote assistance work order. During this phase, the vehicle and the first cloud cabin negotiate and establish an observation link. The observation link refers to the end-to-end communication channel between the first cloud cabin and the vehicle, used to transmit real-time video data collected by the vehicle's cameras to the first cloud cabin. The method for determining successful observation link establishment can refer to the method for determining successful control link establishment described above; to avoid repetition, it will not be repeated here. After receiving the successful observation link establishment information between the vehicle and the first cloud cabin, the scheduling server causes the state machine to jump to the remote control link establishment phase.

[0130] The remote control link establishment phase occurs after the observation link between the vehicle and the first cloud cabin is successfully established. During this phase, the vehicle and the first cloud cabin negotiate to establish a control link. Once the operator confirms that the observation link between the vehicle and the first cloud cabin has been successfully established, and the operator has completed takeover preparations and possesses the observation conditions required for remote control, the dispatch server can respond to the successful establishment of the control link between the vehicle and the first cloud cabin and transition to the control state.

[0131] In this embodiment, by sequentially receiving operator confirmation information sent by the operator through the first cloud cabin and information indicating successful establishment of the observation link between the vehicle and the first cloud cabin, and then responding to information indicating successful establishment of the control link between the vehicle and the first cloud cabin, the system jumps to the control state. This reduces the risk of starting remote control without operator confirmation or observation conditions, thereby improving the safety of remote assistance.

[0132] In some embodiments, the state machine further includes a degradation assignment state, a degradation preparation state, and a degradation control state; the method further includes: In the ready state, in response to the control degradation information sent by the operator through the first cloud cabin, it jumps to the degradation allocation state. In the downgrade allocation state, a work order allocation request is sent to the second cloud cabin. In response to the confirmation information returned by the second cloud cabin, the process jumps to the downgrade preparation state. The first cloud cabin corresponds to the command vehicle control mode, and the second cloud cabin corresponds to the driving vehicle control mode. In the state of preparing for degradation, in response to the successful establishment of the control link between the vehicle and the second cloud cabin, the system jumps to the state of degradation control. In the degraded control state, in response to the control completion information sent by the second cloud cabin, it jumps to the remote control completion state.

[0133] In this embodiment, remote vehicle assistance can be provided through two vehicle control modes: command control mode and driving control mode. Command control mode controls vehicle operation based on semantic control commands that represent behavioral patterns such as "lane change" and "pull over." Driving control mode controls vehicle operation through driving control commands that include specific operational parameters such as steering wheel angle, accelerator pedal opening, brake pedal travel, and gear position.

[0134] In this embodiment, the first cloud cabin corresponds to the command-based vehicle control mode, where the operator can input semantic vehicle control commands via voice, text, or by clicking options. The first cloud cabin then executes vehicle control based on these semantic commands. The second cloud cabin corresponds to the driving-based vehicle control mode, where the operator can input driving control commands via integrated devices such as the steering wheel, accelerator pedal, brake pedal, and gear lever. The second cloud cabin then executes vehicle control based on these driving-based vehicle control commands. In some embodiments, the first and second cloud cabins may only be distinguished at the logical level and may correspond to the same hardware cloud cabin; this embodiment does not limit this.

[0135] Because the control precision and cost of the driving control mode are higher than those of the command control mode, in this embodiment, for a newly created remote assistance work order, a work order allocation request is first sent to the first cloud cabin and the process jumps to the preparation stage. During the preparation stage, if the operator of the first cloud cabin believes that the precision of the command control mode is insufficient to complete the remote assistance, control degradation information can be sent through the first cloud cabin.

[0136] When the dispatch server receives a control degradation message sent by the operator through the first cloud cabin, it can enter the degradation processing flow, release the observation link and resource binding relationship between the first cloud cabin and the vehicle, and cause the state machine to jump to the degradation allocation state. In the degradation allocation state, a work order allocation request is sent to the second cloud cabin. In response to the confirmation message returned by the second cloud cabin, it jumps to the degradation preparation state. In the degradation preparation state, in response to the successful establishment of the control link between the vehicle and the second cloud cabin, it jumps to the degradation control state, thereby remotely assisting the vehicle in a driver-controlled mode. In the degradation control state, in response to the control completion message sent by the second cloud cabin, it jumps to the remote control completion state.

[0137] The transition process between the downgrade allocation state, the downgrade preparation state, and the downgrade control state can refer to the transition process between the allocation state, the preparation state, and the control state described above. To avoid repetition, it will not be repeated here.

[0138] In this embodiment, by setting up two cloud cabins corresponding to different vehicle control modes, when the operator sends control downgrade information through the first cloud cabin corresponding to the vehicle control mode, the downgrade process is initiated, and the second cloud cabin corresponding to the driving control mode provides remote assistance to the vehicle, which can improve the accuracy of remote assistance.

[0139] In some embodiments, the state machine further includes an autonomous state recovery mechanism; the method further includes: In at least one of the allocation state and the preparation state, in response to the autonomous recovery information sent by the vehicle, the system jumps to the autonomous recovery state.

[0140] The process of the dispatch server generating and allocating remote assistance work orders and the cloud cabin preparing to establish a control link takes a certain amount of time. During this process, the vehicle's operating status and the environment in which the vehicle is located may change, so that the vehicle can regain its autonomous capability before the first cloud cabin takes over control, without the need for remote assistance.

[0141] To reduce unnecessary remote takeover operations and to distinguish between the termination state caused by the vehicle's autonomous recovery and the termination state triggered by the operator's confirmation of remote control completion, the state machine in this embodiment also includes an autonomous recovery state. In the autonomous recovery state, the vehicle has regained its autonomous capabilities, thus terminating subsequent remote assistance operations.

[0142] In this embodiment, when a vehicle confirms that it has regained its autonomous capability, it will send autonomous recovery information to the dispatch server, indicating that its autonomous capability has been restored and no remote assistance is required. In response to the autonomous recovery information sent by the vehicle, the dispatch server transitions to the autonomous recovery state.

[0143] After switching to autonomous recovery mode, the scheduling server can generate autonomous recovery event records and store them in the log corresponding to the remote assistance work order. After completing the log archiving, the remote assistance work order will be terminated, thereby reducing unnecessary remote control takeover operations.

[0144] In this embodiment, by responding to the autonomous recovery information sent by the vehicle, the system jumps to the autonomous recovery state, which can reduce unnecessary remote takeover operations.

[0145] In some embodiments, the state machine further includes an abnormal termination state; the method further includes: In at least one of the allocation, preparation, and control states, in response to an abnormal interruption event, the system jumps to the abnormal termination state. An abnormal interruption event includes at least one of the following: the vehicle sending external interference information, the vehicle sending control link interruption information, and the first cloud cabin sending control link interruption information.

[0146] During remote assistance, abnormal interruption events may occur, such as control link anomalies, abnormal vehicle or cloud cabin heartbeats, safety operator intervention, or connection failures, preventing the remote assistance task from continuing. To distinguish between the termination state of a normally completed remote assistance task and the termination state caused by an abnormal interruption event, the state machine in this embodiment also includes an abnormal termination state. In the abnormal termination state, the remote assistance task cannot continue.

[0147] In at least one of the following states: allocation, preparation, and control, the scheduling server can continuously detect whether an abnormal interruption event has occurred. In this embodiment, the abnormal interruption event includes at least one of the following: vehicle sending external interference information, vehicle sending control link interruption information, and first cloud cabin sending control link interruption information. External interference information indicates that the vehicle has been intervened by a safety officer, management platform, etc. The vehicle can send external interference information to the scheduling server when it receives a control command from a subject outside the first cloud cabin, such as a safety officer or management platform. Control link interruption information indicates that the control link between the vehicle and the first cloud cabin is interrupted due to hardware failure, network fluctuations, etc. The vehicle and the first cloud cabin can continuously detect the operating status of the control link through mechanisms such as status reporting and heartbeat information collaboration. When a control link interruption or abnormal operating status is detected, the vehicle sends control link interruption information to the scheduling server.

[0148] Of course, abnormal interruption events can also include other types of events, such as the dispatch server detecting a communication interruption between the dispatch server and the vehicle or the first cloud cabin, etc., which are not limited in this application embodiment.

[0149] In response to an abnormal interruption event, the scheduling server transitions to an abnormal termination state. After transitioning to the abnormal termination state, the scheduling server can record the cause of the abnormality, generate an abnormal event record and store it in the log corresponding to the remote assistance work order, and trigger on-site handling or manual review according to the operational strategy.

[0150] In this embodiment, by maintaining multiple anomaly detection mechanisms and jumping to an abnormal termination state when an abnormal interruption event is detected, it can be distinguished from the normally completed remote control completion state, which facilitates subsequent on-site handling or manual review, while reducing the situation where remote assistance work orders remain in the intermediate state for a long time due to anomalies, and improving the accuracy of state transition.

[0151] In some embodiments, the method further includes: In the controlled state, in response to an abnormal interruption event, a control link disconnect command is sent to the first cloud cabin to cause the first cloud cabin to release the control link between the first cloud cabin and the vehicle.

[0152] In the controlled state, the first cloud module takes over control of the vehicle, and the vehicle receives and executes remote control commands from the first cloud module. If an abnormal interruption event occurs in the controlled state, it indicates a possible anomaly in the control link between the vehicle and the first cloud module, or that the vehicle is no longer able to correctly execute remote control commands. In this case, continuing for the first cloud module to take over control of the vehicle carries a high risk.

[0153] Therefore, in this embodiment of the application, if the scheduling server detects an abnormal interruption event in the controlled state, it sends a control link disconnection command to the first cloud cabin to release the control link between the first cloud cabin and the vehicle, so as to reduce the situation where the vehicle continues to execute erroneous remote control commands under abnormal circumstances.

[0154] In this embodiment, by releasing the control link between the first cloud cabin and the vehicle when an abnormal interruption event occurs in the controlled state, the occurrence of the vehicle executing erroneous remote control commands under abnormal circumstances can be reduced, thereby improving the safety of vehicle operation.

[0155] The following scenario example illustrates the transition process of various states in the state machine of this application embodiment. For example... Figure 2 As shown, the state machine includes the allocation state, the preparation state, the remote control state, the remote control completed state, the autonomous recovery state, and the abnormal termination state.

[0156] After receiving a remote assistance request from a vehicle, the dispatch server generates a remote assistance work order corresponding to the request and sets the state machine for the work order to the "allocation in progress" state. In the "allocation in progress" state, a work order allocation request is sent to the first cloud module. Upon receiving confirmation from the first cloud module, a resource binding record is created between the vehicle, the first cloud module, and the remote assistance work order, and the server transitions to the "preparation in progress" state.

[0157] In the preparation state, the operator sends an operator confirmation message through the first cloud cabin. The first cloud cabin establishes an observation link between the vehicle and the first cloud cabin, followed by a control link between the vehicle and the first cloud cabin. In response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state.

[0158] In the controlled state, the operator inputs remote control commands through the first cloud cabin. The first cloud cabin transmits the remote control commands to the vehicle through the established control link, and the vehicle receives and executes the remote control commands. After confirming that the vehicle has escaped the predicament, the operator sends a control completion message through the first cloud cabin. Upon receiving the control completion message, the system transitions to the remote control completed state. In the remote control completed state, a normal completion record is written to the log corresponding to the remote assistance work order. After the log is archived and the resources of the vehicle, the first cloud cabin, and the control link are released, the remote assistance work order is closed.

[0159] If a vehicle sends autonomous recovery information while in the allocation or preparation states, dispatching orders or preparing for remote assistance will cease, and the system will transition to the autonomous recovery state. In the autonomous recovery state, an autonomous recovery record is written to the log corresponding to the remote assistance work order. After logging is completed and the vehicle, first cloud cabin, and control link resources are released, the remote assistance work order is closed.

[0160] If an abnormal interruption event occurs during the allocation, preparation, or control states, the remote assistance process will stop and transition to the abnormal termination state. If an abnormal interruption event occurs during the control state, a control link disconnect command will be sent to the first cloud cabin to release the control link between the first cloud cabin and the vehicle. In the abnormal termination state, an abnormal termination record will be written to the log corresponding to the remote assistance work order. After logging is completed and the vehicle, first cloud cabin, and control link resources are released, the remote assistance work order will be closed, and on-site handling or manual review will be triggered according to the operational strategy.

[0161] The remote assistance work order state machine control method provided in this application can be executed by a remote assistance work order state machine control device. This application uses the example of a remote assistance work order state machine control device executing the remote assistance work order state machine control method to illustrate the remote assistance work order state machine control device provided in this application.

[0162] This application also provides a remote assistance work order state machine control device.

[0163] like Figure 3 As shown, the remote assistance work order state machine control device includes: The generation module 310 is used to generate a remote assistance work order in response to a remote assistance request from a vehicle, and set the state machine corresponding to the remote assistance work order to the allocation state. The first jump module 320 is used to send a work order allocation request to the first cloud cabin when the allocation is in progress, and jump to the preparation state in response to the cloud cabin confirmation information returned by the first cloud cabin. The second jump module 330 is used to jump to the control state in response to the successful establishment of the control link between the vehicle and the first cloud cabin when in the preparation state. The third jump module 340 is used to jump to the remote control completion state in response to the control completion information sent by the first cloud cabin when in the control state.

[0164] According to the remote assistance work order state machine control device of this application, by designing the remote assistance work order state machine as a three-party state coordination protocol between the cloud server, vehicle and cloud cabin, each state corresponds to a specific consensus stage of the cloud server, vehicle and cloud cabin, and the state transition is triggered by the explicit actions or verifiable events of each participant, the security risks caused by the three-party state deviation can be reduced, thereby improving the security of remote assistance.

[0165] In some embodiments, the remote assistance work order state machine control device further includes a deduplication module, used for: In response to the current message sent by the vehicle, determine the vehicle identifier and semantic type corresponding to the current message; the semantic type includes at least one of assistance request, autonomous recovery, and abnormal event; Based on the vehicle identifier and semantic type corresponding to the current message, the system retrieves information from the historical message database. Based on the retrieved historical messages, the current message is deduplicated. The historical message database stores multiple sets of associations between vehicle identifiers, semantic types, and historical messages.

[0166] In some embodiments, the remote assistance work order state machine control device further includes a verification module, used for: Before performing a state transition, obtain the state change request that triggers the state transition, and determine the current state of the state machine, the original state corresponding to the state change request, the target state corresponding to the state change request, the triggering party of the state change request, and each participant corresponding to the state change request. Jump to the target state if the original state matches the current state, the target state is a valid successor state of the current state, the triggering party has the authority to jump to the state, and all participating parties confirm all the conditions for the jump.

[0167] In some embodiments, the remote assistance work order state machine control device further includes a recording module for: After navigating to the target state, a state change record is generated based on the current time, state change request, current state, target state, triggering party, and participating parties. Status change records are stored in the log corresponding to the remote assistance work order.

[0168] In some embodiments, the generation module 310 is further configured to: generate a remote assistance work order and set the state machine corresponding to the remote assistance work order to a queuing waiting stage; The first jump module 320 is also used for: During the queuing and waiting phase, a work order allocation request is sent to the first cloud cabin, and the process jumps to the work order issuance phase. During the work order issuance phase, in response to the cloud cabin confirmation information sent by the first cloud cabin, the process jumps to the resource binding phase. During the resource binding phase, resource binding records are created for the vehicle, the first cloud cabin, and the remote assistance work order. Once the resource binding records are successfully created, the process will jump to the preparation state.

[0169] In some embodiments, the first jump module 320 is further configured to jump to the queuing waiting stage in at least one of the following situations: During the work order issuance phase, the first cloud cabin returned a cloud cabin rejection message; During the resource binding phase, the creation of the resource binding record failed. While in the preparation state, the first cloud cabin sent a response error message.

[0170] In some embodiments, the first jump module 320 is further configured to: jump to the operator confirmation stage in response to the cloud cabin confirmation information returned by the first cloud cabin; The second jump module 330 is also used for: During the operator confirmation phase, in response to the operator confirmation information sent by the operator through the first cloud cabin, the process jumps to the observation link establishment phase. During the observation link establishment phase, in response to the successful establishment of the observation link between the vehicle and the first cloud cabin, the system jumps to the control link establishment phase. During the control link establishment phase, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state.

[0171] In some embodiments, the remote assistance work order state machine control device further includes a degradation module for: In the ready state, in response to the control degradation information sent by the operator through the first cloud cabin, it jumps to the degradation allocation state. In the downgrade allocation state, a work order allocation request is sent to the second cloud cabin. In response to the confirmation information returned by the second cloud cabin, the process jumps to the downgrade preparation state. The first cloud cabin corresponds to the command vehicle control mode, and the second cloud cabin corresponds to the driving vehicle control mode. In the state of preparing for degradation, in response to the successful establishment of the control link between the vehicle and the second cloud cabin, the system jumps to the state of degradation control. In the degraded control state, in response to the control completion information sent by the second cloud cabin, it jumps to the remote control completion state.

[0172] In some embodiments, the remote assistance work order state machine control device further includes a fourth jump module, used for: In at least one of the allocation state and the preparation state, in response to the autonomous recovery information sent by the vehicle, the system jumps to the autonomous recovery state.

[0173] In some embodiments, the remote assistance work order state machine control device further includes a fifth jump module, used for: In at least one of the allocation, preparation, and control states, in response to an abnormal interruption event, the system jumps to the abnormal termination state. An abnormal interruption event includes at least one of the following: the vehicle sending external interference information, the vehicle sending control link interruption information, and the first cloud cabin sending control link interruption information.

[0174] In some embodiments, the fifth jump module is further configured to: In the controlled state, in response to an abnormal interruption event, a control link disconnect command is sent to the first cloud cabin to cause the first cloud cabin to release the control link between the first cloud cabin and the vehicle.

[0175] The remote assistance work order state machine control device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0176] The remote assistance work order state machine control device in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0177] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described remote assistance work order state machine control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0178] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0179] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described remote assistance work order state machine control method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0180] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0181] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described remote assistance work order state machine control method.

[0182] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described remote assistance work order state machine control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0183] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0186] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0187] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0188] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A remote assistance work order state machine control method, characterized in that, The state machine includes at least an allocation state, a preparation state, a control state, and a remote control completion state; the method includes: In response to a remote assistance request from a vehicle, a remote assistance work order is generated, and the state machine corresponding to the remote assistance work order is set to the allocation state. In the allocation state, a work order allocation request is sent to the first cloud cabin. In response to the cloud cabin confirmation information returned by the first cloud cabin, the process jumps to the preparation state. In the preparation state, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system jumps to the control state. In the controlled state, in response to the control completion information sent by the first cloud cabin, the system jumps to the remote control completion state.

2. The method according to claim 1, characterized in that, The method further includes: In response to a current message sent by the vehicle, determine the vehicle identifier and semantic type corresponding to the current message; the semantic type includes at least one of assistance request, autonomous recovery, and abnormal event; Based on the vehicle identifier and semantic type corresponding to the current message, a search is performed in the historical message database. Based on the retrieved historical messages, the current message is deduplicated. The historical message database stores multiple sets of association relationships between vehicle identifiers, semantic types, and historical messages.

3. The method according to claim 1, characterized in that, The method further includes: Before performing a state transition, obtain the state change request that triggers the state transition, and determine the current state of the state machine, the original state corresponding to the state change request, the target state corresponding to the state change request, the triggering party of the state change request, and each participating party corresponding to the state change request. If the original state matches the current state, the target state is a legitimate successor state of the current state, the triggering party has the authority to jump to the state, and all participating parties confirm all the conditions for the jump, then the jump proceeds to the target state.

4. The method according to claim 3, characterized in that, The method further includes: After navigating to the target state, a state change record is generated based on the current time, the state change request, the current state, the target state, the triggering party, and the participating party. The status change record is stored in the log corresponding to the remote assistance work order.

5. The method according to claim 1, characterized in that, The allocation status includes the queuing waiting stage, the work order issuance stage, and the resource binding stage; The step of generating a remote assistance work order and setting the state machine corresponding to the remote assistance work order to the allocation state includes: generating the remote assistance work order and setting the state machine corresponding to the remote assistance work order to the queuing waiting stage; In the allocation state, a work order allocation request is sent to the first cloud cabin. In response to the cloud cabin confirmation information returned by the first cloud cabin, the process jumps to the preparation state, including: During the queuing and waiting phase, a work order allocation request is sent to the first cloud cabin, and the process jumps to the work order issuance phase. During the work order issuance stage, in response to the cloud cabin confirmation information sent by the first cloud cabin, the process jumps to the resource binding stage; During the resource binding phase, resource binding records are created for the vehicle, the first cloud cabin, and the remote assistance work order. If the resource binding record is successfully created, the process jumps to the preparation state.

6. The method according to claim 5, characterized in that, The method further includes terminating in the queuing phase in at least one of the following situations: During the work order issuance phase, the first cloud cabin returns a cloud cabin rejection message; During the resource binding phase, the creation of the resource binding record failed. In the preparation state, the first cloud cabin sends a response error message.

7. The method according to claim 1, characterized in that, The preparation state includes the operator confirmation stage, the observation link establishment stage, and the control link establishment stage; The step of jumping to the preparation state in response to the cloud cabin confirmation information returned by the first cloud cabin includes: jumping to the operator confirmation stage in response to the cloud cabin confirmation information returned by the first cloud cabin; In the preparation state, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state, including: During the operator confirmation phase, in response to the operator confirmation information sent by the operator through the first cloud cabin, the process jumps to the observation link establishment phase. During the observation link establishment phase, in response to the successful establishment of the observation link between the vehicle and the first cloud cabin, the process jumps to the control link establishment phase. During the control link establishment phase, in response to the successful establishment of the control link between the vehicle and the first cloud cabin, the system transitions to the control state.

8. The method according to claim 1, characterized in that, The state machine further includes a degradation assignment state, a degradation preparation state, and a degradation control state; the method further includes: In the preparation state, in response to the control downgrade information sent by the operator through the first cloud cabin, the system jumps to the downgrade allocation state. In the downgrade allocation state, the work order allocation request is sent to the second cloud cabin. In response to the confirmation information returned by the second cloud cabin, the process jumps to the downgrade preparation state. The first cloud cabin corresponds to the command vehicle control mode, and the second cloud cabin corresponds to the driving vehicle control mode. In the state of preparation for degradation, in response to the successful establishment of the control link between the vehicle and the second cloud cabin, the system jumps to the state of control for degradation. In the degraded control state, in response to the control completion information sent by the second cloud cabin, the system jumps to the remote control completion state.

9. The method according to claim 1, characterized in that, The state machine further includes an autonomous recovery state; the method further includes: In at least one of the allocation state and the preparation state, in response to autonomous recovery information sent by the vehicle, the system jumps to the autonomous recovery state.

10. The method according to claim 1, characterized in that, The state machine also includes an abnormal termination state; The method further includes: In at least one of the allocation state, the preparation state, and the control state, in response to an abnormal interruption event, the process jumps to the abnormal termination state. The abnormal interruption event includes at least one of the following: the vehicle sending external interference information, the vehicle sending control link interruption information, and the first cloud cabin sending control link interruption information.

11. The method according to claim 10, characterized in that, The method further includes: In the controlled state, in response to the abnormal interruption event, a control link disconnection command is sent to the first cloud cabin to cause the first cloud cabin to release the control link between the first cloud cabin and the vehicle.

12. A remote assistance work order state machine control device, characterized in that, include: The generation module is used to generate a remote assistance work order in response to a remote assistance request from a vehicle, and set the state machine corresponding to the remote assistance work order to the allocation state. The first jump module is used to send a work order allocation request to the first cloud cabin in the allocation state, and jump to the preparation state in response to the cloud cabin confirmation information returned by the first cloud cabin. The second jump module is used to jump to the control state in response to the successful establishment of the control link between the vehicle and the first cloud cabin in the preparation state. The third jump module is used to jump to the remote control completion state in response to the control completion information sent by the first cloud cabin in the control state.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the remote assistance work order state machine control method as described in any one of claims 1-11.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the remote assistance work order state machine control method as described in any one of claims 1-11.

Images