Methods, apparatuses, devices, and computer-readable storage media for remote assistance

Abstract

According to embodiments of this disclosure, a method, apparatus, device, and computer-readable storage medium for remote assistance are provided. The method includes: determining a first accessibility attribute of a target area based on perception data of an autonomous vehicle; determining that the first accessibility attribute differs from a second accessibility attribute indicated by map data; in response to determining that the autonomous vehicle is in a stranded state associated with the target area, sending an assistance request to a remote device, the assistance request indicating the extent of the target area; and in response to receiving an indication message from the remote device that passage through the target area is permitted, generating an escape route for traversing the target area. This significantly improves the vehicle's autonomous adaptability in complex road conditions and reduces the risk of vehicle entrapment and traffic disruption.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and more particularly to methods, apparatus, devices, and computer-readable storage media for remote assistance. Background Technology

[0002] Autonomous driving is a technology that uses computers to replace or assist human drivers in perceiving the vehicle's surroundings, planning the vehicle's trajectory, and controlling the vehicle to reach a designated destination.

[0003] During autonomous vehicle operation, unforeseen factors may cause the vehicle to become unable to make autonomous decisions. In such situations, assisting autonomous vehicles in making decisions is a crucial aspect of improving driving safety and reliability. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for remote assistance is provided. The method includes: determining a first accessibility attribute of a target area based on perception data of an autonomous vehicle; determining that the first accessibility attribute differs from a second accessibility attribute indicated by map data; in response to determining that the autonomous vehicle is in a stranded state associated with the target area, sending an assistance request to a remote device, the assistance request indicating the extent of the target area; and in response to receiving an indication message from the remote device that passage through the target area is permitted, generating an escape route for traversing the target area.

[0005] In a second aspect of this disclosure, a remote assistance apparatus is provided. The apparatus includes: a first accessibility attribute determination module configured to determine a first accessibility attribute of a target area based on perception data of an autonomous vehicle; an attribute matching module configured to determine that the first accessibility attribute differs from a second accessibility attribute indicated by map data; an assistance request sending module configured to send an assistance request to a remote device in response to determining that the autonomous vehicle is in a trapped state associated with the target area, the assistance request indicating the extent of the target area; and an escape route generation module configured to generate an escape route for traversing the target area in response to receiving an indication message from the remote device that passage through the target area is permitted.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the methods of the first or second aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the methods of the first or second aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method of the first or second aspect.

[0009] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0012] Figure 2 A flowchart of a method for remote assistance according to some embodiments of the present disclosure is shown;

[0013] Figure 3 A schematic diagram of a help interface according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic diagram illustrating a remote assistance process according to some embodiments of the present disclosure is shown;

[0015] Figure 5 A schematic structural block diagram of a remote assistance apparatus according to some embodiments of the present disclosure is shown; and

[0016] Figure 6 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0021] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0022] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably.

[0023] With the development of autonomous driving technology, autonomous vehicles are increasingly being used in complex road environments. However, current high-definition maps, which serve as a crucial basis for path planning, cannot be updated in real time with dynamic environmental information, such as changes in construction areas, temporary obstacles, or lane markings. This information delay may lead to discrepancies between the high-definition map and the actual situation during actual driving, resulting in path planning errors or traffic obstacles.

[0024] One approach involves using perception modules to identify dynamic changes in the real-world environment, such as using cameras or lidar to detect obstacles. However, the results from these perception modules may suffer from insufficient confidence or false positives. Therefore, how to achieve remote assistance for autonomous vehicles in this situation is a pressing issue that needs to be addressed.

[0025] Embodiments of this disclosure propose a remote assistance method, which may include determining a first accessibility attribute of a target area based on perception data of an autonomous vehicle. The first accessibility attribute is determined to be different from a second accessibility attribute indicated by map data. In response to determining that the autonomous vehicle is in a stranded state associated with the target area, an assistance request is sent to a remote device, the assistance request indicating the extent of the target area. In response to receiving an indication message from the remote device that passage through the target area is permitted, an escape route for traversing the target area is generated.

[0026] The above process identifies differences in the accessibility attributes of target areas by comparing the autonomous vehicle's perception data with map data. When the autonomous vehicle becomes stuck due to an inability to detour, it sends an assistance request to a remote device to indicate the scope of the target area. After receiving the indication message that passage through the target area is permitted, the autonomous vehicle generates an escape route through the target area based on this message, thereby achieving efficient response to dynamic environments, ensuring vehicle accessibility, and reducing the risk of getting stuck. This significantly improves the vehicle's adaptability to complex road conditions, reduces the risk of getting stuck, and increases traffic efficiency.

[0027] Example Environment

[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In environment 100, an assistant 110 can interact directly with a remote device 120, or interact with the remote device 120 via an attached device. The remote device 120 can present a user interface 140 to the assistant 110 for operations such as viewing vehicle information or performing route planning.

[0029] The remote device 120 may include, for example, a cloud device or an edge computing device. In some embodiments, such a remote device 120 may be configured to provide the autonomous vehicle 101 with supplemental perception information about the traffic environment and / or provide guidance for the stranded autonomous vehicle 101 to extricate itself from a difficult situation.

[0030] Environment 100 may include an autonomous vehicle 101. In some embodiments, the autonomous vehicle 101 may be any type of vehicle capable of carrying people and / or objects and moving via a power system such as an engine, including but not limited to cars, trucks, buses, electric vehicles, motorhomes, etc. The autonomous vehicle 101 may be an automated driving vehicle (also referred to as an autonomous vehicle) that integrates functions such as environmental perception, planning and decision-making, and multi-level assisted driving.

[0031] like Figure 1 As shown, the autonomous vehicle 101 is equipped with electronic equipment 150, which can communicate with remote equipment 120. For example, electronic equipment 150 can communicate with remote equipment 120 via appropriate wireless communication methods. Electronic equipment 150 can be any device with computing capabilities, which can generate a route to get out of trouble or execute corresponding commands based on the path planning information issued by remote equipment 120.

[0032] It should be understood that the structure and function of environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0033] Remote assistance methods and processes

[0034] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure. Figure 2 A flowchart of a method 200 for remote assistance according to some embodiments of the present disclosure is shown.

[0035] like Figure 2 As shown in box 201, electronic device 150 determines the first access attribute of the target area based on the perception data of autonomous vehicle 101.

[0036] The autonomous vehicle 101 can acquire real-time environmental data of the target area through the perception module and determine its primary accessibility attribute. As an example, the perception module may include various sensors, such as cameras, LiDAR, ultrasonic sensors (USS), etc., which can work together to provide more accurate environmental perception information.

[0037] The first accessibility attribute can indicate whether the accessibility of the target area is obstructed, the current direction of travel, or speed restrictions. For example, when construction obstacles are detected in the target area, the perception module can identify the specific shape of the target area, the location and type of the obstacles (such as fences or temporary barriers), and mark the area as "impassable". Taking fence recognition as an example, the perception module will perform occupancy network (OCC) recognition on the fence and output its location, contour information, category, and corresponding confidence level. This information may be obtained through a static fusion target detection method. As an example, the electronic device 150 can combine visual data from a camera, precise distance information from a lidar, and the near-field sensing capability of an ultrasonic sensor to ensure the accuracy and reliability of fence recognition. In addition, the first accessibility attribute can also include lane line scenarios. For example, due to lane line changes, the original straight lane line may be changed to a left-turn, right-turn, or U-turn lane line. If the original navigation path is straight travel, the lane corresponding to the changed lane line can also be marked as "impassable".

[0038] In box 202, electronic device 150 determines that the first access attribute is different from the second access attribute indicated by the map data.

[0039] Electronic device 150 compares the first access attribute (generated by the perception module) with the second access attribute stored in the high-precision map data to determine if they are consistent. For example, the first access attribute might indicate that the target area is "impassable," such as when a road is detected to be closed by a construction fence, or when lane markings indicate only left turns while the navigation route indicates going straight. The second access attribute, however, might indicate that the area is "passable" based on map data. If they are inconsistent, for example, if the perception module indicates that the target area is blocked, but the map data does not show any obstacles, then electronic device 150 confirms that the access attributes are different, thus triggering further processing logic.

[0040] In box 203, in response to determining that the autonomous vehicle 101 is in a trapped state associated with a target area, the electronic device 150 sends an assistance request to the remote device 120, the assistance request indicating the extent of the target area.

[0041] Electronic device 150 determines whether autonomous vehicle 101 is in a trapped state associated with the target area. For example, if electronic device 150 fails to plan a detour route that meets safety standards or driving rules based on data collected by the perception module and map data, causing autonomous vehicle 101 to stop before reaching the target area, it can be determined that autonomous vehicle 101 is blocked and unable to detour. Based on this, electronic device 150 can determine that autonomous vehicle 101 is in a trapped state associated with the target area. For example, if the target area is completely closed or the detour route exceeds preset conditions, electronic device 150 can determine that autonomous vehicle 101 is in a trapped state associated with the target area. Preset conditions may include, for example, excessively long detour distances or insufficient safety. For example, insufficient safety may include potential risks in the detour route, such as the detour route being too narrow to allow autonomous vehicle 101 to pass safely, the slope or turning radius of the detour route exceeding the dynamic constraints of autonomous vehicle 101, or the detour route being partially blocked by dynamic obstacles (such as other parked vehicles or pedestrians).

[0042] When it is confirmed that the autonomous vehicle 101 is in a trapped state, the electronic device 150 can generate an assistance request and send it to the remote device 120. This assistance request includes detailed information about the target area, such as representing the location and shape of the target area using polygon coordinates or a bounding box, so that the remote device can accurately identify and address the relevant issues. Taking a construction scenario as an example, the target area can be marked as a closed polygonal region by the perception module. The assistance request can also include the type of obstacle (such as a fence or construction equipment) and its confidence level, so that the remote device can generate targeted guidance.

[0043] In box 204, electronic device 150 generates an escape route for traversing the target area in response to receiving an indication message from remote device 120 that passage through the target area is permitted.

[0044] After receiving an instruction message generated by the remote device 120, the electronic device 150 generates an escape route for traversing the target area based on the information in the message. For example, the instruction message may include multiple waypoints within the target area, which indicate safe passage paths for the vehicle within the target area. The electronic device 150 can calculate a feasible escape route based on the waypoint locations and the dynamic constraints of the autonomous vehicle 101 (such as vehicle width, turning radius, etc.). If the target area is only partially obstructed, such as a construction area with some lanes remaining open, the escape route can point to a safe lane within the target area or guide the vehicle to slow down and pass cautiously, thereby achieving efficient escape.

[0045] Through the above process, the autonomous vehicle 101 achieves rapid response capabilities to dynamic environments. When the traffic attributes of the target area change, the autonomous vehicle 101 can accurately identify problems by comparing perception data with map data, and proactively request remote assistance when stuck. Combined with real-time guidance from remote equipment, the autonomous vehicle 101 can efficiently generate an escape route and resume normal driving, significantly improving its autonomous adaptability in complex road conditions and reducing the risk of vehicle entrapment and traffic disruption.

[0046] As previously mentioned, the instruction message may include multiple waypoints. Based on this, the electronic device 150 can generate an escape route for traversing the target area based on multiple waypoints.

[0047] Waypoints define at least one safe passage path within a target area. Based on the received waypoint information, electronic device 150 can generate an escape route using a path planning algorithm. For example, waypoints can be key reference points within the target area, identifying the specific locations that autonomous vehicle 101 should pass through in sequence to ensure that autonomous vehicle 101 avoids obstacles and successfully traverses the target area. During the planning process, electronic device 150 optimizes the passage path by combining the spatial layout of waypoints, the dynamic constraints of autonomous vehicle 101 (such as minimum turning radius, width restrictions, etc.), and environmental factors (such as the dynamic information of surrounding obstacles and other vehicles).

[0048] Taking a construction scenario as an example, if the target area requires detours due to the closure of some lanes, multiple waypoints in the instruction message can point to the remaining passable lanes. The electronic device 150 generates a smooth escape route based on these waypoints, ensuring that the autonomous vehicle 101 can pass through the construction area smoothly and safely, avoiding further entrapment or passage risks due to improper route planning.

[0049] The following describes the process by which remote device 120 generates instruction information. Based on the assistance request received from autonomous vehicle 101, remote device 120 presents an assistance interface. The assistance interface displays target elements corresponding to the target area. Based on the received preset operation associated with the target element, an instruction message is generated.

[0050] After receiving an assistance request from the autonomous vehicle 101, the remote device 120 will initiate the corresponding processing procedure and generate instruction information. As an example, the remote device 120 can present an assistance interface based on the assistance request. Figure 3A schematic diagram of an assistance interface 300 according to some embodiments of the present disclosure is shown. This assistance interface can display relevant information about a target area. As an example, the relevant information about the target area may include a target element 301 corresponding to the target area. The target area is presented in the form of a target element, such as a polygon or border, to visually mark the area to be processed. Furthermore, the relevant information about the target area may also include the location and extent of the target area identified by combining map components and environmental imagery.

[0051] In the operation area 302 of the assistance interface, operators or trained machine learning models can perform preset operations associated with the target element. These operations may include marking the target area as passable, generating waypoint information, or adjusting the shape and extent of the target area. The remote device 120 generates a final instruction message based on the input from the operator or the trained machine learning model. For example, if the target area is a partially closed construction section, the operator or machine learning model can generate clear passage guidance information by marking passable paths or adding multiple waypoints based on the target element in the assistance interface.

[0052] Ultimately, the generated instruction message will be sent back to autonomous vehicle 101 in digital form, containing key information required for the vehicle's safe passage, such as waypoint coordinates or passage strategies, to ensure that the vehicle can successfully extricate itself from trouble.

[0053] In some embodiments, the preset operation may include a first operation or a second operation. The first operation may be used to mark the target area as a passable area. The second operation may be used to determine multiple waypoints associated with the target area.

[0054] The preset operations provided by remote device 120 can include two operation modes to flexibly respond to different situations in the target area. The two operation modes can correspond to the first operation and the second operation, respectively, which will be introduced separately below.

[0055] The first operation can directly mark the target area as a passable area. When the operator or machine learning model determines that there are no obstacles obstructing passage within the target area or that safe passage conditions are met, it can mark it as "passable" through interactions in the assistance interface (such as clicking or selecting the target area). For example, if the construction fence has been removed or it is only a visual error, the operator or machine learning model can quickly confirm the area status through the first operation and generate an instruction message allowing autonomous vehicle 101 to pass directly.

[0056] The second operation can identify multiple waypoints associated with the target area. When there are complex passage requirements within the target area, such as detouring around obstacles or needing to follow a specific path, operators or machine learning models can set multiple waypoints through the assistance interface to clarify the passage path of autonomous vehicle 101 within the target area. This operation can generate waypoints by dragging or clicking different locations on the map, forming a safe and optimized passage path. For example, when some lanes are closed due to construction, operators or machine learning models can mark the remaining passable routes to ensure that autonomous vehicle 101 safely traverses the target area according to the predetermined trajectory.

[0057] These two operating methods can be flexibly selected according to the actual situation of the target area, thereby improving the efficiency and accuracy of remote assistance. Finally, the remote device 120 generates corresponding instruction messages based on the preset operations performed, providing guidance for the autonomous vehicle 101 to get out of trouble.

[0058] In some embodiments, the assistance interface 300 includes a map component, and the remote device 120 can present target elements corresponding to the target area in the map component.

[0059] The assistance interface 300 may include an integrated map component for visually displaying the spatial information of the target area. The remote device 120 may visualize the target area as target elements in the map component. These target elements may be rendered as polygons, rectangular borders, markers, or other forms to clearly identify the location, extent, and status of the target area.

[0060] During the rendering process, the remote device 120 transforms the data 303 of the target area into intuitive visual elements based on the assistance request sent by the autonomous vehicle 101. For example, for a construction section identified by the perception module, the remote device 120 renders a polygonal outline corresponding to the target area on the map component and distinguishes it by color fill, line style, or marker symbols, indicating that the area is an "area requiring confirmation" or an "impassable area." The rendering process may include dynamically loading map data, drawing the shape of the target area, and overlaying other information from the autonomous vehicle 101 and its surrounding environment.

[0061] Furthermore, the rendered target elements can be combined with other map information (such as road networks, traffic signs, and adjacent areas) to provide operators or machine learning models with comprehensive background information. For example, the map component can dynamically update the relative position of the target area and the autonomous vehicle 101, enabling operators or machine learning models to quickly determine the status of the area and its impact on traffic.

[0062] Through the rendering process, the interface can display the target area in a clear and intuitive way, significantly improving the efficiency and accuracy of operators or machine learning models in handling problems. Furthermore, the map component supports dynamic zooming, panning, and layer switching to facilitate comprehensive analysis of the target area. For example, during confirmation, operators can zoom in on the map to view details of the target area or switch to different view modes (such as satellite view or road view) to obtain clearer background information.

[0063] By presenting target elements in the map component, the assistance interface effectively improves the intuitiveness of information transmission and operational efficiency, providing an intuitive and highly interactive tool for remote assistance.

[0064] In some embodiments, the remote device is further configured to: present an environmental image associated with the autonomous vehicle in the assistance interface, and present an indicator element corresponding to the target area in the environmental image.

[0065] To enhance the efficiency and accuracy of remote operation, in addition to displaying map components in the assistance interface, the remote device 120 can also be configured to display environmental images associated with the autonomous vehicle 101. These environmental images are typically captured in real time by onboard sensors (such as cameras) and can provide a real visual scene of the target area, thereby helping operators or machine learning models to more intuitively understand the current status of the target area.

[0066] For example, when the target area involves a construction zone, the environmental image can clearly show the actual location of the fence, the specific shape of the obstacle, and the layout of its surrounding environment. The remote device 120 will overlay indicator elements corresponding to the target area onto the environmental image. These indicator elements may include polygonal borders, indicator arrows, or highlighted marks to visually display the extent of the target area and its relative position to vehicles.

[0067] Furthermore, the indicator elements can be consistent with the target elements in the map component to ensure consistency in judgments made by operators or machine learning models across different views. For example, target areas marked in the map component will be displayed on the environmental image with the same border or marker. Operators or machine learning models can use the environmental image to confirm the actual accessibility of the target area, such as determining whether a fence has been removed or whether there are obstacles not identified by the perception module.

[0068] By combining environmental imagery with indicator elements, remote device 120 can provide richer contextual information, compensating for the shortcomings of relying solely on map data and ensuring that operators or machine learning models can make quick and accurate judgments in complex environments.

[0069] In some embodiments, the remote device is further configured to update the assistance interface in response to the autonomous vehicle 101 traversing the target area, thereby stopping the presentation of target elements.

[0070] The remote device 120 can also be configured to dynamically update the assistance interface after the autonomous vehicle 101 successfully traverses the target area, stopping the display of target elements related to the target area. This feature is designed to improve the real-time performance and clarity of the assistance interface, preventing irrelevant information from interfering with the operator's subsequent tasks.

[0071] For example, once the autonomous vehicle 101 successfully passes through the target area according to the escape route, the remote device 120 will receive feedback from the autonomous vehicle 101, indicating that the autonomous vehicle 101 has left the target area. Based on this feedback, the assistance interface will automatically remove target elements related to the target area, such as polygon markers in the map component and indicator borders in the environmental image. In addition, the task status related to the target area will be updated to "completed" and removed from the task queue.

[0072] If the autonomous vehicle 101 fails to traverse the target area successfully, for example, due to path planning errors or the appearance of new obstacles and becomes stuck again, the remote device 120 can retain the target elements and regenerate the assistance request to guide the autonomous vehicle 101 to make further adjustments. Through this dynamic update mechanism, the assistance interface can remain simple and efficient, allowing operators to focus on unfinished tasks, reducing unnecessary information interference, and improving the overall response efficiency of the assistance system.

[0073] In some embodiments, presenting a target element corresponding to a target area in the assistance interface includes: determining the shape of the target area based on the assistance request; and drawing the target element corresponding to the shape in the assistance interface.

[0074] When the remote device 120 presents the target element corresponding to the target area in the assistance interface, it first needs to determine the shape of the target area based on the assistance request sent by the autonomous vehicle 101. The shape of the target area can be described by polygon data, rectangles, or other geometric forms provided by the vehicle perception module. This data can include the vertex coordinates or boundary information of the area, used to define the extent of the target area.

[0075] For example, in a road construction scenario, the target area might be an irregular polygon, representing the specific distribution range of the construction fence. The remote device 120 will draw the corresponding target element on the map component or environmental image of the assistance interface based on the shape data included in the assistance request. During drawing, the target element can be highlighted using different styles (such as solid borders, dashed borders, or fill colors) so that operators can quickly identify the area.

[0076] Furthermore, the process of rendering the target element can include dynamic shape adjustments. For example, if the perceptual data of the target area contains updated information, the remote device 120 will recalculate and redraw the shape of the target element to keep it consistent with the latest data. This dynamic adjustment ensures that the display of the target area is highly synchronized with the actual situation.

[0077] By accurately drawing the shape of the target area in the assistance interface, the remote device 120 provides operators with an intuitive and accurate reference tool, which helps to improve the efficiency and accuracy of remote assistance, especially in scenarios with complex or irregular target areas.

[0078] In some embodiments, in response to a mismatch between the accessibility attributes of at least one area indicated by the perception data and the map data, the electronic device 150 sends a change confirmation message to the remote device 120. Based on the change response message received from the remote device, the perception data and / or map data of the autonomous vehicle 101 are updated.

[0079] When electronic device 150 detects a mismatch between the accessibility attributes of at least one area indicated by the sensing data and the high-precision map data, it sends a change confirmation message to remote device 120 to verify the accuracy of the accessibility attributes. This message may include the location information of the target area, attribute data provided by the sensing module (such as accessibility status and obstacle type), and a description of the differences between the data and the map data, to facilitate the judgment of remote device 120.

[0080] After receiving the change confirmation message, remote device 120 can verify the actual situation of the target area in various ways. For example, it can combine real-time environmental images, historical data, or manual judgment by operators. Based on the verification results, remote device 120 can generate a change response message. The change response message will include the confirmed accessibility information of the target area, such as whether the updated status is "accessible" or "inaccessible," as well as supplementary details.

[0081] Upon receiving a change response message, electronic device 150 updates the perception data and / or map data of autonomous vehicle 101 based on the message's content. For example, if the change response message confirms that the perception module's identification is accurate, autonomous vehicle 101 adds or modifies the accessibility attributes of the relevant target area in the map data. If the perception module's results contain errors, map data or correction information from remote devices is used first to ensure the accuracy of subsequent route planning and accessibility decisions.

[0082] This change confirmation and data update mechanism significantly improves the adaptability of the autonomous vehicle 101 in dynamic environments. For example, in construction scenarios, when the fence layout does not match the map data, the autonomous vehicle 101 can quickly correct the data through this mechanism, enabling more reliable decision-making. Ultimately, this solution ensures a high degree of consistency between perception and map data, providing more stable driving support for the autonomous vehicle 101.

[0083] Example process

[0084] Figure 4 A schematic diagram of a remote assistance process 400 according to some embodiments of the present disclosure is shown. Process 400 can be implemented at an electronic device 150. References are made below. Figure 1 Describe the process 400.

[0085] In box 401, electronic device 150 performs reality change detection on the target area through the perception module, obtains real-time information of the target area, such as obstacle type, location and related access attributes, and thus determines the first access attribute.

[0086] In box 402, electronic device 150 compares the first access attribute with the second access attribute stored in the high-precision map to determine if there is a difference between the two. For example, the perception module indicates that the target area is "impassable," while the map data shows "accessible."

[0087] In box 403, electronic device 150 assesses the confidence level of the discrepancy results to confirm whether the perceived data can support a reliable decision. If the confidence level is not met, electronic device 150 sends a message containing the discrepancy information to remote device 120 in box 404. The discrepancy information may typically include the extent of the target area, differences in accessibility attributes, and relevant obstacle information.

[0088] In box 405, remote device 120 initiates a real-time change processing procedure based on the received discrepancy information to ensure accurate judgment and efficient handling of the problem.

[0089] In box 406, remote device 120 renders the target area in the assistance interface based on the difference information, such as visually displaying the location and shape of the target area through map components or environmental images, and marking the differences in its accessibility attributes.

[0090] In box 407, remote device 120 confirms the rendered discrepancies through operator judgment or machine learning model analysis. The confirmation results may include changes to the target area (e.g., new obstacles or roads returning to normal) or specific escape routes.

[0091] For escape routes, a fallback option can also be included. A brief explanation follows, using a fence as an example. For instance, for a fence inside the shoulder or outside the lane boundary line, if the fence is newly added, the electronic equipment 150 can control the autonomous vehicle 101 to pass normally, using the fence as a reference boundary and maintaining a lateral distance of at least 20cm. If the fence is removed, the electronic equipment 150 can control the autonomous vehicle 101 to pass normally, using the shoulder as a reference boundary.

[0092] For fences within an intersection, if the fence is newly added, the electronic device 150 can control the autonomous vehicle 101 to pass normally, provided it does not affect the virtual lane; if it affects the virtual lane, the passable area can be reduced, with the fence as the boundary. If the fence is removed, the passable area expands, and the electronic device 150 can control the autonomous vehicle 101 to use the shoulder as the boundary.

[0093] For situations along lane lines (100m before intersections) on roads, if a fence is newly added, the road link at the fence can be closed and lane changes restricted. If it affects lane selection at the intersection, proceed according to the lane; otherwise, change lanes in advance to a lane without a fence. If a fence is removed, and the road link remains unchanged, normal traffic can proceed within the lane, and the lane change restrictions in the original fenced area will be lifted.

[0094] For left / right turns and U-turns (not at an intersection but with a gap before the intersection, causing the original traffic rules to be cancelled), if the fence is removed and the road connection is changed, autonomous vehicle 101 must change lanes to an unaffected lane to detour.

[0095] For left / right turns and U-turns (not at an intersection but with a gap before the intersection, and the gap does not cause the original traffic rules to be cancelled), the original traffic rules can be maintained.

[0096] For situations along lane lines (excluding intersections) on the road, if a barrier has been newly added, vehicles have priority to change lanes to lanes without barriers. If the lane change fails, vehicles can proceed normally within the original barrier area. If all lanes have barriers, vehicles have priority to enter lanes with normal traffic flow. If a barrier has been removed, vehicles can proceed normally within the original barrier area, and lane change restrictions in the original barrier area are lifted.

[0097] If a barrier occupies more than 30cm of lane space (100m before the intersection), and this is due to a newly added barrier, change lanes in advance if the lateral space occupies more than 30cm. If changing lanes in advance is not possible, proceed slowly within the lane and initiate remote lane assist. If the lateral space is less than 30cm, proceed cautiously around the obstacle or change lanes without affecting lane selection at the intersection. If the barrier has been removed, proceed cautiously if it affects lane selection at the intersection. If it does not affect lane selection at the intersection, change lanes first; if changing lanes is not possible, proceed cautiously.

[0098] When a barrier occupies lane space (excluding intersections), if it's a newly added barrier and the lateral space occupies more than 30cm, change lanes in advance. If changing lanes in advance is not possible, proceed slowly within the lane and initiate remote assistance. If the lateral space is less than 30cm, prioritize changing lanes; if changing lanes is not possible, proceed cautiously around the obstacle. If the barrier has been removed, prioritize changing lanes; if changing lanes is not possible, proceed cautiously with communication.

[0099] In box 408, remote device 120 sends the confirmed change information back to autonomous vehicle 101 to guide subsequent decision-making and execution. Furthermore, if in box 403, the confidence level of the difference assessment result by electronic device 150 has reached the required standard, it indicates that the perception module of autonomous vehicle 101 has sufficient basis for judging the differences in the traffic attributes of the target area and can directly confirm the change information of the target area. In this case, electronic device 150 does not need to request remote device intervention, but directly outputs the change information in box 408, passing the confirmed changes (such as the addition or removal of obstacles, or updates to traffic status) to subsequent modules for processing, thereby accelerating the decision-making process and reducing reliance on remote assistance.

[0100] In box 409, electronic device 150 updates its internal map data based on the change information to ensure that subsequent path planning and environmental model are consistent with the actual situation.

[0101] In box 410, electronic device 150 performs a fusion calculation of the passage strategy based on the received escape route and the vehicle's own attributes.

[0102] Finally, in box 411, the vehicle completes the determination of the passage strategy and executes the passage task according to the generated strategy, successfully passing through the target area.

[0103] Example devices and equipment

[0104] Figure 5 A schematic structural block diagram of a remote assistance device 500 according to certain embodiments of the present disclosure is shown. The device 500 may be implemented as or included in an electronic device 150. Various modules / components in the device 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0105] As shown in the figure, the device 500 includes a first accessibility attribute determination module 501, configured to determine a first accessibility attribute of a target area based on perception data from the autonomous vehicle. An attribute matching module 502 is configured to determine that the first accessibility attribute differs from a second accessibility attribute indicated by map data. An assistance request sending module 503 is configured to send an assistance request to a remote device in response to determining that the autonomous vehicle is in a trapped state associated with the target area, the assistance request indicating the extent of the target area. An escape route generation module 504 is configured to generate an escape route for traversing the target area in response to receiving an indication message from a remote device that passage through the target area is permitted.

[0106] In some embodiments, the instruction message includes multiple waypoints, and the escape route generation module 504 can be specifically configured to generate an escape route for traversing the target area based on the multiple waypoints.

[0107] In some embodiments, the instruction message is generated based on the following process: a remote device presents an assistance interface based on an assistance request received from an autonomous vehicle; a target element corresponding to the target area is presented in the assistance interface; and an instruction message is generated based on a preset operation associated with the target element.

[0108] In some embodiments, the preset operation includes: a first operation or a second operation, wherein the first operation is used to mark the target area as a passable area; and the second operation is used to determine a plurality of waypoints associated with the target area.

[0109] In some embodiments, the assistance interface includes a map component, and presenting target elements corresponding to the target area in the assistance interface includes presenting target elements corresponding to the target area in the map component.

[0110] In some embodiments, the remote device is further configured to: present an environmental image associated with the autonomous vehicle in the assistance interface; and present indicator elements corresponding to the target area in the environmental image.

[0111] In some embodiments, the remote device is further configured to update the assistance interface in response to the autonomous vehicle traversing the target area, thereby stopping the presentation of target elements.

[0112] In some embodiments, presenting a target element corresponding to a target area in the assistance interface includes: determining the shape of the target area based on the assistance request; and drawing the target element corresponding to the shape in the assistance interface.

[0113] In some embodiments, the apparatus 500 further includes a data update module. The data update module is configured to send a change confirmation message to a remote device in response to a mismatch between the accessibility attributes of at least one area indicated by the perception data and the map data. Based on the change response message received from the remote device, the perception data and / or map data of the autonomous vehicle are updated.

[0114] Figure 6 A block diagram is shown illustrating a computing device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that... Figure 6 The computing device 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The computing device 600 shown can be used to implement Figure 1 Remote devices 120 or electronic devices 150.

[0115] like Figure 6 As shown, computing device 600 is in the form of a general-purpose computing device. Components of computing device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage devices 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 600.

[0116] Computing device 600 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to computing device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 600.

[0117] The computing device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0118] The communication unit 640 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 600 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 600 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0119] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 600 can also communicate as needed with one or more external devices (not shown) via communication unit 640. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 600, or with any device (e.g., network card, modem, etc.) that enables computing device 600 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0120] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0121] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0122] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0123] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0125] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for remote assistance, comprising: Based on the perception data of autonomous vehicles, the primary accessibility attribute of the target area is determined; It is determined that the first access attribute is different from the second access attribute indicated by the map data; In response to determining that the autonomous vehicle is in a trapped state associated with the target area, an assistance request is sent to a remote device, the assistance request indicating the extent of the target area; as well as In response to receiving an indication message from the remote device that passage through the target area is permitted, an escape route for traversing the target area is generated.

2. The method according to claim 1, wherein the indication message includes multiple waypoints, and generating an escape route for traversing the target area includes: Based on the multiple waypoints, an escape route for traversing the target area is generated.

3. The method of claim 1, wherein the indication message is generated based on the following process: The remote device presents an assistance interface based on the assistance request received from the autonomous vehicle; The target element corresponding to the target area is displayed in the assistance interface; as well as The instruction message is generated based on the received preset operation associated with the target element.

4. The method according to claim 3, wherein the preset operation includes: A first operation, wherein the first operation is used to mark the target area as a passable area; or The second operation is used to determine multiple waypoints associated with the target area.

5. The method of claim 3, wherein the assistance interface includes a map component, and presenting target elements corresponding to the target area in the assistance interface includes: The target element corresponding to the target area is presented in the map component.

6. The method of claim 5, wherein the remote device is further configured to: The assistance interface displays an environmental image associated with the autonomous vehicle; and The environmental image displays indicator elements corresponding to the target area.

7. The method of claim 3, wherein the remote device is further configured to: In response to the autonomous vehicle traversing the target area, the assistance interface is updated to stop displaying the target element.

8. The method of claim 3, wherein presenting the target element corresponding to the target area in the assistance interface includes: Based on the assistance request, the shape of the target area is determined; as well as Draw the target element corresponding to the shape in the assistance interface.

9. The method according to claim 1, further comprising: In response to a mismatch between the accessibility attributes of at least one area indicated by the sensing data and the map data, a change confirmation message is sent to the remote device; as well as Based on the change response message received from the remote device, the perception data and / or map data of the autonomous vehicle are updated.

10. A remote assistance device, comprising: The first access attribute determination module is configured to determine the first access attribute of the target area based on the perception data of the autonomous vehicle. The attribute matching module is configured to determine that the first access attribute is different from the second access attribute indicated by the map data; An assistance request sending module is configured to send an assistance request to a remote device in response to determining that the autonomous vehicle is in a stranded state associated with the target area, the assistance request indicating the extent of the target area; as well as The escape route generation module is configured to generate an escape route for traversing the target area in response to receiving an indication message from the remote device that passage through the target area is permitted.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 9.

Images