Unmanned vehicle control method and device, electronic equipment and storage medium

Abstract

The invention discloses an unmanned vehicle control method and device, electronic equipment and a storage medium method, relates to the technical field of unmanned vehicle control and is applied to a remote control system.The control method comprises the steps that in-vehicle monitoring data of an unmanned vehicle and environment data of the position where the unmanned vehicle is located are obtained, the in-vehicle monitoring data indicates behaviors or states of passengers in the unmanned vehicle; determining a passenger state and a passenger risk level of the unmanned vehicle according to the in-vehicle monitoring data and the environment data; and determining a care scheme according to the passenger state and the risk level, and controlling the unmanned vehicle based on the care scheme. According to the method and the device, the passengers can be cared in the whole process in an unmanned driving scene, the fundamental conversion from passive monitoring to active care of the passengers in the vehicle is realized, meanwhile, personalized service measures can be automatically triggered according to the states of the passengers, and the sense of safety and comfortable experience of the passengers are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of control of unmanned vehicles, and more specifically, to a control method, apparatus, electronic device and storage medium for unmanned vehicles. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, the Occupant Monitoring System (OMS), as a crucial in-vehicle intelligent system, has become an important technological means to improve vehicle active safety. Currently, OMS technology primarily utilizes sensing devices such as visual sensors and millimeter-wave radar to construct a multi-layered occupant status monitoring system, thereby achieving functions such as occupant presence detection, location recognition, and behavior perception. However, existing OMS systems mainly remain at the passive monitoring level, unable to provide personalized services based on occupant status and struggling to achieve intelligent proactive intervention, thus failing to meet the needs of providing comprehensive occupant care in autonomous driving scenarios. Therefore, how to provide comprehensive occupant care in autonomous driving scenarios has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of this application propose a control method, apparatus, electronic device, and storage medium for an unmanned vehicle to improve the above-mentioned problems.

[0004] According to a first aspect of the embodiments of this application, a control method for an autonomous vehicle is provided for use in a remote control system. The method includes: acquiring in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of occupants inside the autonomous vehicle; determining the occupant state and the risk level of the occupants based on the in-vehicle monitoring data and the environmental data; determining a care plan based on the occupant state and the risk level; and controlling the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when an anomaly occurs to the occupants inside the autonomous vehicle.

[0005] In some embodiments, the remote control system includes a monitoring client, and controlling the autonomous vehicle based on the care scheme includes: if the care scheme is a remote care scheme, generating warning information based on the occupant status and the risk level; controlling the autonomous vehicle to decelerate based on the warning information, and sending the warning information to the monitoring client.

[0006] In some embodiments, the method further includes: if the care scheme is a vehicle-side care scheme, then controlling the playback of music in the autonomous vehicle according to the vehicle-side care scheme.

[0007] In some embodiments, after controlling the autonomous vehicle based on the care scheme, the method further includes: receiving a monitoring instruction from the monitoring client; determining control parameters in the monitoring instruction; and controlling the autonomous vehicle according to the control parameters.

[0008] In some embodiments, after receiving the monitoring instruction from the monitoring client, the method further includes: determining the monitoring type according to the monitoring instruction; if the monitoring type is real-time monitoring, then sending the real-time monitoring data of the autonomous vehicle to the monitoring client; if the monitoring type is keyframe playback, then determining keyframe monitoring data from the in-vehicle monitoring data according to the target timestamp corresponding to the monitoring instruction, and sending the keyframe monitoring data to the monitoring client.

[0009] In some embodiments, determining the occupant status and risk level of the autonomous vehicle based on the in-vehicle monitoring data and the environmental data includes: performing behavior recognition on the in-vehicle monitoring data to determine the occupant's behavior status; if the behavior status indicates that the occupant has engaged in abnormal behavior, analyzing the abnormal behavior determined by the environmental data to determine the cause of the abnormal behavior; and determining the occupant status and risk level based on the cause.

[0010] In some embodiments, after controlling the autonomous vehicle based on the care scheme, the method further includes: if it is determined that the occupant's state has changed, then determining that the anomaly has been resolved; obtaining the execution log of the care scheme, the timeline corresponding to the execution log, and the handling result of the care scheme, and generating a handling report based on the execution log, the timeline, and the handling result; and sending the handling report to the monitoring client.

[0011] According to a second aspect of the embodiments of this application, a control device for an autonomous vehicle is provided, applied to a remote control system. The device includes: a first acquisition module, configured to acquire in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle; a risk level determination module, configured to determine the occupant state and the risk level of the occupants based on the in-vehicle monitoring data and the environmental data; and a first control module, configured to determine a care plan based on the occupant state and the risk level, and control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when an abnormality occurs to the occupants inside the autonomous vehicle.

[0012] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the control method for an unmanned vehicle as described above is implemented.

[0013] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the control method of the unmanned vehicle as described above.

[0014] In this application, the occupant status and risk level of the autonomous vehicle are first determined based on the acquired in-vehicle monitoring data and environmental data of the vehicle's location. Then, a care plan is determined based on the occupant status and risk level, and the autonomous vehicle is controlled accordingly. This application enables full-process care for occupants in autonomous driving scenarios, achieving a fundamental shift from passive monitoring to proactive care. Furthermore, it can automatically trigger personalized service measures based on occupant status, significantly improving occupant safety and comfort.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the embodiments of this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] Figure 1 This is a schematic diagram of a remote control system according to an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating a control method for an unmanned vehicle according to an embodiment of this application.

[0019] Figure 3 This is a flowchart illustrating a control method for an unmanned vehicle according to another embodiment of this application.

[0020] Figure 4 This is a flowchart illustrating a control method for an unmanned vehicle according to another embodiment of this application.

[0021] Figure 5This is a display interface of a monitoring client according to an embodiment of the present application.

[0022] Figure 6 This is a schematic flowchart illustrating a control method for an unmanned vehicle according to another embodiment of this application.

[0023] Figure 7 This is a schematic flowchart illustrating a control method for an unmanned vehicle according to another embodiment of this application.

[0024] Figure 8 This is an occupant care method that integrates autonomous driving and remote monitoring, as shown in an embodiment of this application.

[0025] Figure 9 This is a block diagram of a control device for an unmanned vehicle according to an embodiment of this application.

[0026] Figure 10 This is a hardware structure diagram of an electronic device according to an embodiment of this application.

[0027] The accompanying drawings have illustrated specific embodiments of the present application. More detailed descriptions will follow. These drawings and descriptions are not intended to limit the scope of the present application's embodiments in any way, but rather to illustrate the concepts of the present application's embodiments to those skilled in the art through specific embodiments. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0029] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0030] Please see Figure 1 , Figure 1 This application illustrates a remote control system provided by an embodiment of the present application, such as... Figure 1 As shown below, an exemplary description of the control method for realizing unmanned vehicles using a remote control system will be provided.

[0031] In one alternative implementation, the remote control system includes a vehicle-side server 110, a cloud server 120, and a monitoring client 130. The vehicle-side server 110 and cloud server 120 refer to hardware devices or computing units, while the monitoring client 130 refers to a software unit or module.

[0032] For example, the cloud server 120 obtains the in-vehicle monitoring data and environmental data of the autonomous vehicle's location uploaded by the vehicle-side server 110. Then, based on the in-vehicle monitoring data and environmental data, it determines the occupant status and risk level of the autonomous vehicle. Next, it determines a care plan based on the occupant status and risk level, and then sends the care plan to the vehicle-side server 110, thereby controlling the autonomous vehicle based on the care plan.

[0033] For example, the vehicle-side server 110 controls the perception module 111, data preprocessing module 112, and communication gateway 113 in the autonomous vehicle; the cloud server 120 includes a cloud data management center 121 and an intelligent analysis and decision-making unit 122. The vehicle-side server 110 collects multimodal data through the perception module 111 and preprocesses the multimodal data through the data preprocessing module 112. The perception module 111 includes sensors such as high-definition cameras and millimeter-wave radar, used to capture occupant behavior data (such as video streams, posture, and facial expressions) and environmental data (such as temperature and humidity) in real time. After the behavior data and environmental data are preprocessed (filtered, compressed, and encrypted) by the data preprocessing module 112, they are uploaded to the cloud server 120 via the communication gateway 113 based on a 5G / V2X network.

[0034] The cloud server 120 can be used to perform data fusion, analysis, and decision-making functions. The intelligent analysis and decision-making unit 122 uses deep learning algorithms to perform behavior recognition and risk assessment on the received data and generates tiered response strategies. The monitoring client 130 provides a remote interactive interface via software or web client, enabling users or guardians to monitor the real-time status of occupants, and simultaneously providing functions such as receiving alerts, two-way voice communication, and issuing commands.

[0035] First, the perception module 111 in the vehicle-side server 110 collects multimodal data of the occupants (such as behavioral videos and environmental parameters), and uploads it to the cloud server 120 after encryption by the data preprocessing module 123. After receiving the data, the cloud server 120 performs verification, standardization and scene recognition (such as recognizing the scene of a child riding alone in a car). The monitoring client 130 establishes a connection channel with the cloud server 120 by successfully subscribing to the service.

[0036] Then, the cloud server 120 performs multi-source data fusion analysis (such as using convolutional neural networks to identify abnormal behavior), assesses the occupant's risk level (such as a level 2 alarm), and generates a personalized care plan (such as playing music or notifying family members), which is then sent to the vehicle-side server 110 and the monitoring client 130. Upon receiving the care plan, the client 130 generates and displays a status summary and warning prompts. The cloud server 120 allocates network bandwidth to the vehicle-side server 110 and the monitoring client 130 based on the care plan for voice communication. Family members can remotely intervene through the monitoring client 130, sending control commands to the cloud server 120, which forwards the commands to the vehicle-side server 110. The vehicle-side server 110 executes the received control commands and controls the autonomous vehicle based on the care plan.

[0037] Finally, after executing the instruction (such as vehicle deceleration), the vehicle-side server 110 sends the execution result back to the cloud server 120; the cloud server 120 determines the execution effect, generates a handling report, and uses the data to optimize the algorithm model in the cloud server 120, while sending the handling report to the monitoring client 130.

[0038] Figure 1 The system in [the document] can be used to implement the following Figure 2 For the described control method of the autonomous vehicle, please refer to [link / reference]. Figure 2 , Figure 2 This application illustrates a control method for an autonomous vehicle according to an embodiment of the present application. In a specific embodiment, this control method for an autonomous vehicle can be applied to, for example... Figure 9 The control device 700 of the unmanned vehicle shown and the electronic device 800 equipped with the control device 700 of the unmanned vehicle are shown. Figure 10 The specific process of this embodiment will be described below. Of course, it is understood that this method can be executed by an electronic device with computing capabilities, such as a cloud server, an edge server with computing capabilities, or other processors. The following will focus on... Figure 2 The process shown is described in detail and applied to a remote control system. The control method for the unmanned vehicle may specifically include the following steps 210-230.

[0039] Step 210: Obtain in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle.

[0040] As an alternative approach, various types of sensors can be installed in autonomous vehicles to collect in-vehicle monitoring data and environmental data about the vehicle's location. The vehicle's on-board server can then upload this data to a cloud server, which can directly access and use this data to identify the behavior or state of the occupants.

[0041] In one optional scenario, after the autonomous vehicle's various sensors collect different types of sensor data, the vehicle-side server first preprocesses this data before sending it to a cloud server connected to the vehicle-side server. Optionally, the vehicle-side server can classify the different types of sensor data to obtain in-vehicle monitoring data and environmental data. This data is then filtered, compressed, and encoded, followed by encryption to obtain encrypted data packets, which are then sent to the cloud server connected to the vehicle-side server. Optionally, the vehicle-side server can send the encrypted data packets to the cloud server via a 5G network or vehicle-to-everything (V2X) communication.

[0042] Optionally, after receiving the encrypted sensor data, the cloud server first performs decoding and decryption processing to obtain the in-vehicle monitoring data and environmental data of the autonomous vehicle's location. The sensors in the autonomous vehicle may include image sensors, millimeter-wave radar sensors, lidar sensors, infrared sensors, temperature sensors, and humidity sensors, etc. The in-vehicle monitoring data may be real-time behavioral or physiological data of the occupants inside the vehicle. For example, information related to the behavior or state of the occupants inside the autonomous vehicle, such as their facial expressions, posture, body temperature, respiratory rate, heart rate, and image data of the occupants inside the vehicle; environmental data may include the temperature, humidity, and real-time traffic conditions at the location of the autonomous vehicle.

[0043] As an alternative approach, after a user initiates a viewing request on a client connected to the cloud server, the cloud server sends a data acquisition command to the vehicle-side server based on the viewing request. The vehicle-side server, which is connected to the cloud server, then collects in-vehicle monitoring data and environmental data of the autonomous vehicle's location based on the data acquisition command.

[0044] Step 220: Determine the occupant status and risk level of the driverless vehicle based on the in-vehicle monitoring data and the environmental data.

[0045] As an alternative approach, after acquiring in-vehicle monitoring data and environmental data, the occupant status and risk level of the autonomous vehicle can be determined by identifying the in-vehicle monitoring data and environmental data, thereby enabling the autonomous vehicle to be controlled based on the occupant status and risk level.

[0046] In one alternative scenario, a pre-trained recognition neural network can be set up in the cloud server. In-vehicle monitoring data can be sent to the recognition neural network for recognition to determine the reference behavior and reference state of the occupants in the autonomous vehicle. Then, combined with environmental data, it can be determined whether the occupants' reference behavior or reference state is related to the environment, thereby determining the occupant's state and risk level in the autonomous vehicle.

[0047] Optionally, after determining the occupant status of the autonomous vehicle based on in-vehicle monitoring data and environmental data, the risk level of the occupant in the event of an accident in the autonomous vehicle can be determined based on the mapping relationship between the occupant status and risk level. To ensure rapid and accurate determination of the occupant's risk level, different mapping relationships between occupant status and risk level can be pre-set. For example, if the occupant status is an emergency state, its corresponding risk level is Level 1, which represents the highest level of urgency.

[0048] Step 230: Determine a care plan based on the occupant status and the risk level, and control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when the occupant experiences an anomaly in the autonomous vehicle.

[0049] As an alternative approach, after determining the occupant status and risk level, the cloud server can first determine a care plan for the occupants based on the occupant status and risk level, and then send the care plan to the vehicle-side server that is connected to the cloud server. In this way, the vehicle-side server can control the autonomous vehicle according to the received care plan, thereby ensuring the safety and comfort of the occupants in the autonomous vehicle.

[0050] In one alternative scenario, to avoid wasting vehicle resources or compromising passenger safety or comfort by using the same care plan regardless of the passenger's condition, different care plans can be determined based on different passenger conditions and risk levels. Optionally, to proactively and adaptively care for passengers in different conditions, different care strategies corresponding to different passenger conditions can be pre-set, and then different care instructions can be set in different care strategies according to the passenger's risk level to obtain a specific care plan.

[0051] In another alternative scenario, to ensure the accuracy of the determined care plan, the scene information of the autonomous vehicle can be determined first based on in-vehicle monitoring data. Then, the care plan can be determined by combining the occupant status, risk level, and scene information. For example, if the scene information indicates that a child is traveling alone and the occupant status indicates that the child is crying, with a risk level of level two, a care plan of slowing down and playing a children's song can be generated. This care plan is then sent to the vehicle-side server, which controls the vehicle to slow down and activates the autonomous vehicle's audio-visual system to play a children's song.

[0052] In the embodiments of this application, the occupant status and risk level of the autonomous vehicle are first determined based on the acquired in-vehicle monitoring data and environmental data of the autonomous vehicle's location. Then, a care plan is determined based on the occupant status and risk level, and the autonomous vehicle is controlled based on this care plan. This application enables full-process care for occupants in autonomous driving scenarios, achieving a fundamental shift from passive monitoring to proactive care for in-vehicle occupants. Simultaneously, it can automatically trigger personalized service measures based on the occupant status, significantly improving the occupant's sense of security and comfort.

[0053] Please see Figure 3 , Figure 3 This application illustrates a control method for an unmanned vehicle according to an embodiment of this application. The following will focus on... Figure 3 The process shown is described in detail and applied to a remote control system, which includes a monitoring client. The control method for the unmanned vehicle may specifically include the following steps 310-350.

[0054] Step 310: Obtain in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle.

[0055] Step 320: Determine the occupant status and risk level of the driverless vehicle based on the in-vehicle monitoring data and the environmental data.

[0056] Step 330: Determine a care plan based on the occupant status and the risk level.

[0057] The specific steps of steps 310-330 can be found in steps 210-230, and will not be repeated here.

[0058] Step 340: If the care plan is a remote care plan, then generate early warning information based on the occupant status and the risk level.

[0059] As an alternative approach, when the care plan is determined to be a remote care plan, it can be determined that the user or guardian needs to intervene to comfort the occupants in the vehicle. In this case, in order to ensure that the user or guardian can communicate with the occupants in the vehicle or that the user or guardian can understand the status of the occupants in the vehicle in real time, an early warning message can be generated based on the occupant status and the occupant's risk level. This early warning message can then be used to control the autonomous vehicle.

[0060] In one optional scenario, when the care plan is determined to be a remote care plan, in order to avoid panic or fear among the occupants and lead to excessive behavior, warning information including the need for the autonomous vehicle to slow down or stop immediately can be generated based on the occupant's status and risk level. The vehicle-side server can then control the autonomous vehicle to slow down or stop based on this warning information.

[0061] Step 350: Based on the warning information, control the unmanned vehicle to decelerate and send the warning information to the monitoring client.

[0062] As an alternative approach, to ensure accurate control of autonomous vehicles, after generating corresponding warning information, the cloud server can generate a control strategy based on this information and then send the control strategy to the vehicle-side server. The vehicle-side server can then generate control commands based on this strategy, enabling it to control the autonomous vehicle to decelerate. To ensure successful transmission of the control strategy, the cloud server uses the vehicle-side server identifier corresponding to the in-vehicle monitoring data received to send the control strategy to the target vehicle-side server. This avoids safety incidents caused by the inability to control the autonomous vehicle in a timely manner due to failed control strategy transmission.

[0063] In one optional scenario, to ensure that users or guardians can promptly understand the real-time status of the occupants in the vehicle, after the cloud server generates an early warning message, the message can be sent to a monitoring client connected to the cloud server. Upon receiving the warning message, the monitoring client can issue an alarm based on the message, ensuring that guardians or users can promptly view the status of the occupants through the monitoring client and then reassure or remotely control the autonomous vehicle based on the occupants' condition.

[0064] Optionally, to ensure the monitoring client receives warning information promptly, the user or guardian can establish a communication connection with the monitoring client using the license plate number of the autonomous vehicle or by connecting the monitoring client directly to the vehicle. After the cloud server sends the warning information to the monitoring client, the monitoring client can request permission from the cloud server to view the monitoring data of the autonomous vehicle based on the warning information. The cloud server then sends the in-vehicle monitoring data to the monitoring client according to the permission request, thereby ensuring that the user or guardian can view the in-vehicle monitoring data on the monitoring client to understand the occupants' status in the vehicle in real time.

[0065] In some embodiments, after step 330, the method further includes: if the care scheme is a vehicle-side care scheme, then controlling the music playback of the autonomous vehicle according to the vehicle-side care scheme.

[0066] As an alternative approach, when the care plan is determined to be a vehicle-side care plan, it can be determined that the care plan can be executed through sensors or interactive devices in the vehicle-side system of the autonomous vehicle, so that the occupants in the vehicle can receive proactive care, thereby achieving a synergistic improvement in occupant safety and riding experience.

[0067] In one alternative scenario, after generating a vehicle-side care plan, the cloud server can send the plan to the vehicle-side server, which is connected to the cloud server. Upon receiving the vehicle-side care plan, the vehicle-side server generates vehicle-side control commands based on the plan and then sends the vehicle-side control commands to the corresponding actuators, so that the actuators execute the corresponding vehicle-side control commands to complete the vehicle-side care plan.

[0068] Optionally, the vehicle-side care solution is used to soothe occupants by playing music in the autonomous vehicle when the risk level is low, thereby ensuring their emotional well-being and improving their riding experience. Optionally, the vehicle-side care solution also includes generating control settings for the autonomous vehicle's lights, climate control, and windows based on environmental data. This allows the vehicle-side server to control these functions, further enhancing the occupant's riding experience.

[0069] In this embodiment, when a remote care plan is determined, early warning information is generated based on the occupant status and risk level. Then, the autonomous vehicle is controlled to decelerate based on the early warning information, and the early warning information is sent to the monitoring client. This ensures that family members or guardians can participate in the monitoring process in real time, forming a human-machine collaborative proactive care mechanism, which effectively enhances the transparency and participation of the monitoring process.

[0070] Please see Figure 4 , Figure 4This application illustrates a control method for an unmanned vehicle according to an embodiment of this application. The following will focus on... Figure 4 The process shown is described in detail and applied to a remote control system, which includes a monitoring client. The control method for the unmanned vehicle may specifically include the following steps 410-450.

[0071] Step 410: Obtain in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle.

[0072] Step 420: Determine the occupant status and risk level of the driverless vehicle based on the in-vehicle monitoring data and the environmental data.

[0073] Step 430: Determine a care plan based on the occupant status and the risk level, and control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when the occupant experiences an anomaly in the autonomous vehicle.

[0074] The specific steps of steps 410-430 can be found in steps 210-230, and will not be repeated here.

[0075] Step 440: Receive monitoring instructions from the monitoring client.

[0076] As an alternative, users or guardians can monitor the real-time status of occupants inside the vehicle through a monitoring client. Users can trigger a monitoring request through the monitoring interface of the monitoring client, which will generate a monitoring command and send the command to the cloud server. The cloud server can then send the corresponding monitoring data back to the monitoring client and control the autonomous vehicle based on the monitoring command.

[0077] In one optional scenario, to ensure the driving safety of the autonomous vehicle, before the monitoring client sends monitoring instructions, the user or guardian can subscribe to the autonomous vehicle formation service in the monitoring client, place an order in the monitoring client to generate subscription order information, and then send the subscription order information to the cloud server. The cloud server verifies the monitoring client based on the received subscription order information. After successful verification, a secure connection channel is established between the cloud server and the monitoring client, enabling information exchange between the monitoring client and the cloud server.

[0078] Optionally, the subscription order information includes the license plate number of the autonomous vehicle, passenger information, and verification information. Then, the cloud server determines whether it has received a trip order with the same license plate number and passenger information from the trip information sent by the vehicle server based on the license plate number and passenger information. If it is determined that such a trip order exists, it is verified through the verification information.

[0079] As an alternative approach, the monitoring client's display interface includes a status display area, a function operation area, and a primary information prompt area, such as... Figure 5 As shown, users or guardians can monitor the occupants' status inside the vehicle in real time through the status display area and view the historical monitoring data of the occupants inside the vehicle through the status display area; users or guardians can also control the autonomous vehicle and perform other emergency handling functions through the different function controls displayed in the function operation area.

[0080] Optionally, the function operation area includes multiple function controls such as call, deceleration, air conditioning, music, and alarm. Users or guardians can control the autonomous vehicle or directly trigger an alarm by clicking or dragging the corresponding function controls. Figure 5 As shown, when the call function control is triggered, the monitoring client can send a communication request command to the cloud server. The cloud server then allocates dedicated bandwidth for voice calls to both the monitoring client and the vehicle server based on this command. This allows the guardian or user to communicate with the occupants in the vehicle using the dedicated bandwidth allocated by the monitoring client and the vehicle server. The deceleration, air conditioning, and music functions control the autonomous vehicle's deceleration, air conditioning temperature adjustment, and music playback, respectively. The alarm function control allows the user or guardian to send an alarm request to the cloud server. Based on this request, the cloud server sends an alarm command to the vehicle server, which then directly alerts the alarm platform, simultaneously informing the platform of the autonomous vehicle's current location.

[0081] Optionally, the monitoring client can also provide real-time status alerts for occupants based on warning information sent by the cloud server. For example... Figure 5 As shown, when a passenger falls inside the vehicle, the monitoring client displays a notification message: "Latest Alert: Passenger Fall Detected." Optionally, to ensure that users or guardians can know the passenger's real-time location, the monitoring client can also display the current location of the autonomous vehicle and the distance between the autonomous vehicle and the monitoring client.

[0082] Step 450: Determine the control parameters in the monitoring command, and control the unmanned vehicle according to the control parameters.

[0083] As an alternative approach, after receiving monitoring commands from the monitoring client, the cloud server can first analyze the commands to determine the control parameters within them. These control parameters can then be used to control the autonomous vehicle. These control parameters may include those for smoothly decelerating the vehicle and adjusting the air conditioning to a comfortable temperature.

[0084] Optionally, the monitoring command may include parameters for controlling the driving speed of the autonomous vehicle and parameters for controlling the interactive or output devices of the autonomous vehicle. In order to ensure the accuracy of the control of the autonomous vehicle, the control parameters in the monitoring command are first determined, and then the corresponding control command is generated based on the control parameters. The control command is then sent to the vehicle-side server that is connected to the cloud server, so that the vehicle-side server can control the autonomous vehicle according to the control command.

[0085] In some embodiments, after step 440, the method further includes: determining the monitoring type according to the monitoring instruction; if the monitoring type is real-time monitoring, then sending the real-time monitoring data of the autonomous vehicle to the monitoring client; if the monitoring type is keyframe playback, then determining keyframe monitoring data from the in-vehicle monitoring data according to the target timestamp corresponding to the monitoring instruction, and sending the keyframe monitoring data to the monitoring client.

[0086] As an alternative approach, after the cloud server receives the monitoring instruction sent by the monitoring client, in order to ensure that the user or guardian can promptly view or understand the specific situation of the passenger after boarding the vehicle, the monitoring type corresponding to the monitoring instruction can be determined first. In this way, the corresponding monitoring data can be sent to the monitoring client based on the monitoring type for the user or guardian to view.

[0087] In one optional scenario, the user or guardian generates a corresponding monitoring instruction through the corresponding monitoring control on the display interface of the monitoring client. The monitoring instruction includes an identifier corresponding to the monitoring type triggered by the user or guardian through the corresponding monitoring control. In this way, the cloud server can directly determine the corresponding monitoring type based on the identifier in the received monitoring instruction.

[0088] Optionally, when the monitoring type is determined to be real-time monitoring, it can be determined that the user or guardian currently needs to view the real-time data of the occupants inside the vehicle. In this way, the cloud server can send the video data uploaded in real time by the vehicle server to the monitoring client, so that the user or guardian can view the real-time video of the occupants inside the vehicle in the monitoring client.

[0089] Optionally, if the monitoring type is determined to be keyframe playback, then the user or guardian needs to view keyframe data showing abnormal behavior of the occupant inside the vehicle. Therefore, after receiving and identifying the in-vehicle monitoring data uploaded by the vehicle-side server, the cloud server identifies the keyframe data showing abnormal behavior of the occupant inside the vehicle based on the identification results. The corresponding keyframe data is then sent to the monitoring client, allowing the user or guardian to view the keyframe data on the monitoring client.

[0090] Optionally, the status display area of ​​the monitoring client's display interface includes a real-time monitoring video window and a keyframe playback window, such as... Figure 5 As shown, users or guardians can view real-time monitoring video or keyframe video of occupants inside the autonomous vehicle by clicking the real-time monitoring video window or keyframe playback window. After a user or guardian clicks the real-time monitoring video window or keyframe playback window, the monitoring client can generate a monitoring command carrying a real-time identifier or keyframe identifier based on the triggered real-time monitoring video window or keyframe playback window, thereby ensuring that the cloud server can determine the corresponding monitoring type based on the received monitoring command.

[0091] In this embodiment, monitoring parameters are determined according to the monitoring instructions from the monitoring client, thereby enabling the autonomous vehicle to be controlled based on these monitoring parameters. This solves the problem of the lack of an effective remote interaction mechanism for monitoring occupants and the inability of family members to participate in the monitoring process, significantly improving the occupants' sense of security and comfort.

[0092] Please see Figure 6 , Figure 6 This application illustrates a control method for an unmanned vehicle according to an embodiment of this application. The following will focus on... Figure 6 The process shown is described in detail and applied to a remote control system, which includes a monitoring client. The control method for the unmanned vehicle may specifically include the following steps 510-550.

[0093] Step 510: Obtain in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle.

[0094] Step 520: Perform behavior recognition on the in-vehicle monitoring data to determine the occupant's behavioral state.

[0095] As an optional approach, to ensure passenger safety and a positive passenger experience, the cloud server, upon receiving in-vehicle monitoring data and environmental data, can first perform behavioral recognition on the in-vehicle monitoring data to determine the passenger's behavioral state. Specifically, a pre-trained CNN recognition model is configured on the cloud server, and this model is used to identify the in-vehicle monitoring data, thereby obtaining the passenger's behavior inside the vehicle and determining their behavioral state. The passenger's behavioral state includes both abnormal and normal behavior.

[0096] Step 530: If the behavioral status indicates that the occupant has engaged in abnormal behavior, then the abnormal behavior of the occupant is analyzed based on the environmental data to determine the cause of the abnormal behavior.

[0097] As an alternative approach, after obtaining the occupant's behavioral status, and when the behavioral status indicates that the occupant is behaving abnormally, to avoid accidents during the operation of the autonomous vehicle and to prevent a poor passenger experience when no intervention is needed, the abnormal behavior can first be analyzed based on environmental data to determine the cause of the abnormal behavior. For example, in excessively high temperatures, occupants may exhibit abnormal behaviors such as increased heart rate, increased breathing rate, or constant shifting of position; in this case, the cause of the abnormal behavior can be determined to be excessively high temperature.

[0098] Step 540: Determine the occupant status and the risk level based on the stated cause.

[0099] As an alternative approach, after determining the cause, the occupant's condition or risk level can be determined based on the specific type of cause. This specific type can include environmental factors, medical conditions, and the autonomous vehicle's driving characteristics. For example, if the cause is determined to be an environmental factor, the occupant's condition can be determined to be normal, with a low risk level. In this case, care can be provided to the occupant by adjusting the temperature, brightness, and air circulation within the autonomous vehicle. If the cause is determined to be a driving characteristic of the autonomous vehicle, the occupant's condition can be determined to be normal, with a medium risk level. In this case, care can be provided to the occupant by adjusting the autonomous vehicle's speed, route, or acceleration. If the cause is determined to be a medical condition, the occupant's condition can be determined to be abnormal or dangerous, with a high risk level. In this case, it is necessary to notify the occupant's guardian for human intervention and responsible care.

[0100] Step 550: Determine a care plan based on the occupant status and the risk level, and control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when the occupant experiences an anomaly in the autonomous vehicle.

[0101] For a detailed description of steps 510 and 550, please refer to steps 210 and 230, which will not be repeated here.

[0102] In this embodiment, the in-vehicle monitoring data is first used to identify the occupant's behavior status. When the behavior status indicates that the occupant is behaving abnormally, the abnormal behavior is analyzed based on the environmental data to determine the cause of the abnormal behavior. Based on the cause, the occupant's status and risk level are determined, ensuring the accuracy of the determined care plan.

[0103] Please see Figure 7 , Figure 7 This application illustrates a control method for an unmanned vehicle according to an embodiment of this application. The following will focus on... Figure 7 The process shown is described in detail and applied to a remote control system. The remote control system includes a monitoring client, and the control method for the unmanned vehicle may specifically include the following steps 610-660.

[0104] Step 610: Obtain in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle.

[0105] Step 620: Determine the occupant status and risk level of the driverless vehicle based on the in-vehicle monitoring data and the environmental data.

[0106] Step 630: Determine a care plan based on the occupant status and the risk level, and control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when the occupant experiences an anomaly in the autonomous vehicle.

[0107] The specific steps of steps 610-630 can be found in steps 210-230, and will not be repeated here.

[0108] Step 640: If it is determined that the occupant status has changed, then the anomaly is resolved.

[0109] As an optional approach, when actively controlling or intervening in autonomous vehicles based on a care-based solution, the vehicle-side server continuously uploads real-time data collected from occupants inside the vehicle to a cloud server. The cloud server then uses this continuously uploaded data to continuously identify the occupant's status until a change in status indicates the anomaly has been resolved. Optionally, to ensure occupant safety, after determining that an abnormal occupant status has changed from an abnormal state to a normal state, the cloud server waits for a preset time after which the occupant's status remains unchanged, thus confirming the anomaly has been resolved. Optionally, the cloud server uses the continuously uploaded data from the vehicle-side server to determine whether the occupant's behavioral indicators have returned to normal range, thus determining whether an abnormal encounter occurred.

[0110] Step 650: Obtain the execution log of the care plan, the timeline corresponding to the execution log, and the handling result of the care plan, and generate a handling report based on the execution log, the timeline, and the handling result.

[0111] As an optional approach, after confirming that the anomaly has been resolved, in order to ensure that the cloud server can learn from each instance of proactive care provided to the occupants, and to ensure that users or guardians can be informed of changes in the occupants' status in real time, the execution log of the care plan, the timeline corresponding to the execution log, and the handling results of the care plan can be obtained. This allows for the generation of corresponding handling reports based on the execution log, timeline, and handling results.

[0112] In one alternative scenario, after the vehicle-mounted server controls the autonomous vehicle based on the received care plan, it can generate execution feedback based on the control result of the autonomous vehicle and upload the execution feedback to the cloud server. The cloud server can then analyze the execution feedback to determine the handling result of the care plan.

[0113] Optionally, after identifying abnormal contact, the cloud server can generate an execution log and a timeline of the execution log for the autonomous vehicle based on the received in-vehicle monitoring data, environmental data, care plan, and execution feedback.

[0114] Optionally, after obtaining the execution log, the timeline of the execution log, and the handling results from the cloud server, a handling report can be generated. This handling report may include the timestamp of the passenger's abnormal behavior, the data indicating the passenger's abnormal behavior, the care plan for controlling the autonomous vehicle, the timestamp and control parameters for controlling the autonomous vehicle based on the care plan, the timeline and control parameters for the monitoring client's intervention to control the autonomous vehicle, the timeline of the passenger's abnormality resolution, and the data indicating the passenger's abnormality resolution, etc.

[0115] Step 660: Send the handling report to the monitoring client.

[0116] As an alternative, to ensure that users or guardians can fully understand the entire process of passengers riding in driverless vehicles, the cloud server can send the handling report to the monitoring client after generating the handling report, so that users or guardians can understand the entire process of passengers riding in driverless vehicles through the monitoring client.

[0117] Optionally, to further improve the passenger experience and safety of autonomous vehicles via the cloud server, after receiving the incident report, the algorithm for determining the care plan in the cloud server can be optimized and learned based on the data optimization algorithm model of the incident report. For example, the threshold for adjusting the sitting posture can be adjusted to ensure that the care measures are implemented and the system continues to learn. Specifically, after the monitoring client receives the incident report, the information prompt area of ​​the monitoring client can be updated to a green status "Command executed successfully," and a complete report can be pushed to parents for viewing. The report content focuses on changes in behavioral data.

[0118] In this embodiment, after controlling the autonomous vehicle based on the care plan, if it is determined that the occupant status has changed, the anomaly can be determined to be resolved. Then, the execution log of the care plan, the timeline corresponding to the execution log, and the handling result of the care plan are obtained. A handling report is generated based on the execution log, timeline, and handling result. Finally, the handling report is sent to the monitoring client to ensure that the care measures are implemented and the system continues to learn.

[0119] Figure 8 This is an example of an embodiment of the occupant care method integrating autonomous driving and remote monitoring, as shown in this application. Figure 8 As shown, the vehicle-side server controls the autonomous vehicle to continuously collect multimodal data (such as video streams, vital signs, and behavioral data) of the occupants, and performs real-time analysis on the video streams to extract key behavioral features (such as body posture angles and facial expression change frequency) to obtain occupant behavioral data and autonomous vehicle environmental data. Then, the environmental and behavioral data are compressed and encrypted, and then the encrypted behavioral and environmental data are packaged and uploaded to the cloud server connected to the vehicle-side server via the 5G-V2X network.

[0120] After receiving the encrypted behavioral and environmental data uploaded by the vehicle-side server, the cloud server first decrypts and standardizes the multimodal behavioral and environmental data to obtain standardized data. Then, the standardized data is used for identification to determine the current care scenario of the autonomous vehicle. The current care scenario includes care scenarios such as children riding alone or the elderly traveling.

[0121] Parents or guardians can subscribe to this trip service through the remote monitoring client. They submit order information and request access permissions to the cloud server through the client. After receiving the request, the cloud data management center on the cloud server authenticates the identity by matching the order information, vehicle information, and passenger information. Once authentication is successful, a secure connection channel is established between the cloud server and the vehicle server, and the real-time monitoring video is forwarded to the remote monitoring client.

[0122] The intelligent analysis and decision-making unit in the cloud server performs multimodal fusion analysis based on behavioral and environmental data uploaded from the vehicle-side server to determine whether a passenger has exhibited abnormal behavior (such as falling, fainting, or crying). Abnormal behavior can be identified using deep learning models (such as CNNs) with a confidence level greater than 90%. When abnormal behavior is determined, environmental data is used for auxiliary analysis (e.g., the vehicle's interior temperature is normal, ruling out environmental factors). Simultaneously, considering the current care scenario, a care plan is retrieved from the cloud server's adaptive strategy library to determine the specific care plan. For example, in a scenario where an 8-year-old child is traveling alone in an autonomous vehicle from school home, a distance of 5 kilometers with an estimated travel time of 15 minutes, the specific care plan could be to initiate a two-way voice call for reassurance, smoothly reduce the vehicle speed to 40 km / h to improve comfort, and notify the parents to intervene.

[0123] After the care plan is determined, the cloud server generates a status summary and warning information for the occupants and sends the occupants' status summary and warning information to the remote monitoring client, while sending the care plan to the vehicle-side server.

[0124] After receiving the status summary and warning information, the remote client sends a call request to the cloud server. This call request can be either a voice call or a video call. Based on this call request, the cloud server can allocate communication resources to the remote client and the vehicle server, thereby ensuring that the guardian or user can communicate with the occupants in the autonomous vehicle through the allocated communication resources.

[0125] Meanwhile, guardians or users can trigger control commands to slow down or turn back the vehicle in the remote monitoring client and send the control commands to the cloud server, which will then forward the control commands to the vehicle server.

[0126] After receiving the control command, the vehicle-mounted server controls the autonomous vehicle according to the control command. After the vehicle-mounted server executes the control command, it returns the execution completion status to the cloud server and uploads the autonomous vehicle's status data and related data of the executed control command.

[0127] After receiving the completion status, the cloud server forwards it to the remote monitoring client. Based on the received status data and relevant mechanical data analysis of the control commands, the cloud server generates a handling report and optimization suggestions. These suggestions are then used to optimize the care plan. Furthermore, the cloud server forwards the handling report (including a timeline, execution logs, and effect evaluation) to the remote monitoring client. Users or caregivers can view the report and provide feedback and suggestions through the remote monitoring client. The remote monitoring client then uploads this feedback and suggestions to the cloud server, which can then further optimize the plan based on the feedback and suggestions.

[0128] The above embodiments describe in detail the control method for an autonomous vehicle provided in this application. In other embodiments, this application also provides a control device for an autonomous vehicle. Figure 9 This is a block diagram of a control device for an unmanned vehicle according to an embodiment of this application, such as... Figure 9 As shown, the control device 700 for the unmanned vehicle, which is applied to the remote control system, includes: a first acquisition module 710, a risk level determination module 720, and a first control module 730.

[0129] The first acquisition module 710 is used to acquire in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle; the risk level determination module 720 is used to determine the occupant state and the risk level of the occupants of the autonomous vehicle based on the in-vehicle monitoring data and the environmental data; the first control module 730 is used to determine a care plan based on the occupant state and the risk level, and control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when the occupants experience an anomaly inside the autonomous vehicle.

[0130] In some embodiments, the remote control system includes a monitoring client, and the first control module 730 includes: a warning information generation unit, configured to generate warning information based on the occupant status and the risk level if the care plan is a remote care plan; and an information sending unit, configured to control the unmanned vehicle to decelerate based on the warning information and send the warning information to the monitoring client.

[0131] In some other embodiments, the first control module 730 further includes a playback unit, configured to control the playback of music in the autonomous vehicle according to the vehicle-side care scheme if the care scheme is a vehicle-side care scheme.

[0132] In some embodiments, the control device 700 of the unmanned vehicle further includes: a receiving module for receiving monitoring instructions from the monitoring client; and a second control module for determining control parameters in the monitoring instructions and controlling the unmanned vehicle according to the control parameters.

[0133] In other embodiments, the control device 700 of the autonomous vehicle further includes: a monitoring type determination module, used to determine the monitoring type according to the monitoring instruction; a data transmission module, used to send the real-time monitoring data of the autonomous vehicle to the monitoring client if the monitoring type is real-time monitoring; and a key frame monitoring data determination module, used to determine key frame monitoring data in the in-vehicle monitoring data according to the target timestamp corresponding to the monitoring instruction if the monitoring type is key frame playback, and send the key frame monitoring data to the monitoring client.

[0134] In some embodiments, the risk level determination module 720 includes: a behavior state determination unit, configured to perform behavior recognition on the in-vehicle monitoring data to determine the behavior state of the occupant; an analysis unit, configured to analyze the abnormal behavior of the occupant based on the environmental data if the behavior state indicates that the occupant has engaged in abnormal behavior, and determine the cause of the abnormal behavior; and a risk level determination unit, configured to determine the occupant's status and the risk level based on the cause.

[0135] In some other embodiments, the control device 700 of the unmanned vehicle further includes: a second determining module, used to determine that the abnormality is resolved if it is determined that the occupant's state has changed; a handling report generation module, used to obtain the execution log of the care plan, the timeline corresponding to the execution log, and the handling result of the care plan, and generate a handling report based on the execution log, the timeline, and the handling result; and a report sending module, used to send the handling report to the monitoring client.

[0136] According to one aspect of the embodiments of this application, an electronic device is also provided, such as... Figure 10 As shown, the electronic device 800 also includes a processor 810 and one or more memories 820. The one or more memories 820 are used to store computer-readable instructions executed by the processor 810. When the processor 810 executes the program instructions, it implements the above-described control method for the unmanned vehicle.

[0137] Furthermore, the processor 810 may include one or more processing cores. The processor 810 runs or executes instructions, programs, code sets, or instruction sets stored in the memory 820, and retrieves data stored in the memory 820. Optionally, the processor 810 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 810 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor and may be implemented using a separate communication chip.

[0138] According to one aspect of this application, a computer-readable storage medium is also provided, which may be included in the cloud server described in the above embodiments; or it may exist independently and not assembled into the cloud server. The aforementioned computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0139] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0140] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0141] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0142] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A control method for an unmanned vehicle, characterized in that, Applied to a remote control system, the method includes: Acquire in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle; Based on the in-vehicle monitoring data and the environmental data, the occupant status and risk level of the autonomous vehicle are determined. A care plan is determined based on the occupant's condition and the risk level, and the autonomous vehicle is controlled based on the care plan. The care plan is an emergency response plan for when the occupant experiences an anomaly inside the autonomous vehicle.

2. The method according to claim 1, characterized in that, The remote control system includes a monitoring client, and the control of the autonomous vehicle based on the care scheme includes: If the care plan is a remote care plan, then an early warning message is generated based on the occupant status and the risk level; Based on the warning information, the driverless vehicle is controlled to decelerate, and the warning information is sent to the monitoring client.

3. The method according to claim 2, characterized in that, The method further includes: If the care solution is a vehicle-side care solution, then the music played in the autonomous vehicle is controlled according to the vehicle-side care solution.

4. The method according to claim 1, characterized in that, After controlling the autonomous vehicle based on the care scheme, the method further includes: Receive monitoring instructions from the monitoring client; The control parameters in the monitoring command are determined, and the driverless vehicle is controlled according to the control parameters.

5. The method according to claim 4, characterized in that, After receiving the monitoring instructions from the monitoring client, the method further includes: The monitoring type is determined according to the monitoring instructions; If the monitoring type is real-time monitoring, then the real-time monitoring data of the unmanned vehicle will be sent to the monitoring client. If the monitoring type is keyframe playback, then the keyframe monitoring data is determined from the in-vehicle monitoring data according to the target timestamp corresponding to the monitoring instruction, and the keyframe monitoring data is sent to the monitoring client.

6. The method according to claim 1, characterized in that, The step of determining the occupant status and risk level of the autonomous vehicle based on the in-vehicle monitoring data and the environmental data includes: Behavior recognition is performed on the in-vehicle monitoring data to determine the occupant's behavioral state; If the behavioral status indicates that the occupant has engaged in abnormal behavior, then the abnormal behavior of the occupant is analyzed based on the environmental data to determine the cause of the abnormal behavior. The occupant status and risk level are determined based on the stated reasons.

7. The method according to any one of claims 1-6, characterized in that, After controlling the autonomous vehicle based on the care scheme, the method further includes: If it is determined that the occupant's status has changed, then the anomaly is determined to be resolved; Obtain the execution log of the care plan, the timeline corresponding to the execution log, and the handling result of the care plan, and generate a handling report based on the execution log, the timeline, and the handling result; The handling report is sent to the monitoring client.

8. A control device for an unmanned vehicle, characterized in that, The device, used in a remote control system, includes: The first acquisition module is used to acquire in-vehicle monitoring data of the autonomous vehicle and environmental data of the location of the autonomous vehicle, wherein the in-vehicle monitoring data indicates the behavior or state of the occupants inside the autonomous vehicle. The risk level determination module is used to determine the occupant status and the occupant's risk level based on the in-vehicle monitoring data and the environmental data. The first control module is used to determine a care plan based on the occupant status and the risk level, and to control the autonomous vehicle based on the care plan, wherein the care plan is an emergency response plan for when the occupant experiences an anomaly in the autonomous vehicle.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images