Equipment movement control method, vehicle, computer equipment and storage medium

Abstract

The invention provides an equipment movement control method, a vehicle, computer equipment and a storage medium, and the method comprises the steps: obtaining behavior description information issued by a cloud; the behavior description information is used for describing information of at least one target location where the equipment needs to arrive; controlling the equipment to move to the at least one target location based on the behavior description information and the surrounding environment information sensed by the equipment; wherein in the moving process of the equipment, an emergency control instruction issued by a cloud end is received, and the emergency control instruction is responded; the behavior description information and the emergency control instruction are generated by the cloud based on environmental data collected by each sensing device deployed in an environment where the device is located. According to the embodiment of the invention, the capability of normal operation of the equipment in the environment where the equipment is located is improved in a mode of combining autonomous control of the equipment with cloud remote braking.

Description

Technical Field

[0001] The present disclosure relates to the technical field of device control, and in particular to a device movement control method, a vehicle, a computer device, and a storage medium. Background Art

[0002] With the improvement of intelligent technology, various autonomous mobile devices have emerged, such as logistics robots, humanoid robots, robot dogs, sweeping robots, and intelligent vehicles with autonomous driving functions. After these devices are rolled off the production line, they need to be tested in various scenarios. Due to the autonomous positioning function of the equipment, they can move autonomously to a certain extent in the test scenario. For example, autonomous driving vehicles can operate in multiple scenarios such as closed road test scenarios, urban road test scenarios, parking lot test scenarios, and factory test scenarios based on their own sensors.

[0003] Due to the complex environment in some test scenarios, the autonomous mobility capabilities of the above-mentioned equipment are usually unable to meet the requirements for mobile safety, flexibility, etc. in these complex environments. For example, the current vehicle autonomous driving technology is usually unable to meet the requirements for the level of autonomous driving in factory testing scenarios, especially unable to deal well with abnormal situations that occur when the vehicle is running in the factory.

[0004] Therefore, how to control the normal operation of automatic mobile equipment in complex test scenarios is an issue worthy of attention. Summary of the invention

[0005] The embodiments of the present disclosure at least provide a device movement control method, a vehicle, a computer device, and a storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides a device movement control method, which is applied to a device capable of autonomous movement, comprising:

[0007] Obtaining behavior description information sent from the cloud; the behavior description information is used to describe information of at least one target location that the device needs to reach;

[0008] Based on the behavior description information and the surrounding environment information perceived by the device, control the device to move to the at least one target location;

[0009] Among them, during the movement of the device, the emergency control instructions issued by the cloud are received, and the emergency control instructions are responded to; the behavior description information and the emergency control instructions are generated by the cloud based on the environmental data collected by various sensing devices deployed in the environment where the device is located.

[0010] In an optional embodiment, the method further includes:

[0011] When it is determined based on the surrounding environment information that there is an obstacle in the moving direction of the device, controlling the device to stop moving and wait;

[0012] In response to the obstacle not being removed after the set waiting time, sending obstacle reminder information to the cloud;

[0013] Receive a first instruction issued by the cloud based on the obstacle reminder information; the first instruction is used to control the device to continue waiting until the obstacle is removed, or to control the device to move along the route replanned by the cloud, or to control the device to move to the target location replanned by the cloud.

[0014] In an optional implementation, the behavior description information includes the order of arrival at multiple target locations;

[0015] In the case where the first instruction is used to control the device to move to the target location re-planned by the cloud, the method further includes:

[0016] After the device reaches the target location re-planned by the cloud, the device is controlled to sequentially reach the remaining target locations that have not been reached according to the arrival order of the multiple target locations included in the behavior description information; or,

[0017] After the device arrives at the target location re-planned in the cloud, the device is controlled to arrive at the remaining unreached target locations in sequence according to the re-planned arrival order indicated by the first instruction.

[0018] In an optional implementation, during the movement of the device, the method further includes:

[0019] When it is determined based on the surrounding environment information that there is an obstacle in the moving direction of the device, controlling the device to autonomously avoid the obstacle;

[0020] If the device re-plans the moving route based on the obstacle avoidance requirement, the re-planned moving route is sent to the cloud;

[0021] After receiving confirmation information from the cloud regarding the re-planned moving route, the device is controlled to move according to the re-planned moving route.

[0022] In an optional implementation, the behavior description information includes movement strategies corresponding to a plurality of perception scenarios respectively; different movement strategies correspond to different perception scenarios;

[0023] The controlling the device to move to the at least one target location based on the behavior description information and the surrounding environment information perceived by the device includes:

[0024] Determine a current target perception scene based on the surrounding environment information perceived by the device; the target perception scene belongs to one of the multiple perception scenes;

[0025] Controlling the movement of the device according to the target movement strategy for the target perception scenario indicated by the behavior description information.

[0026] In an optional implementation, the behavior description information further includes description information of a target behavior that needs to be performed after arriving at the target location; the target behavior is related to the requirements corresponding to the target location;

[0027] The method further comprises:

[0028] After arriving at the target location, the target behavior is executed based on the description information of the target behavior.

[0029] In an optional embodiment, the method further includes:

[0030] After the device arrives at the target location, executing the operation instructions of the staff at the target location;

[0031] The operation instruction is at least one of a voice instruction, a gesture instruction or a remote control instruction issued by a remote control device.

[0032] In an optional implementation, the behavior description information is described in natural language;

[0033] The controlling the device to move to the at least one target location based on the behavior description information and the surrounding environment information perceived by the device includes:

[0034] Performing natural language understanding on the behavior description information, and parsing the behavior description information into executable control instructions;

[0035] The device is controlled to move according to the control instruction and the surrounding environment information sensed by the device.

[0036] In a second aspect, the embodiment of the present disclosure further provides a device movement control method, which is applied to a cloud server, comprising:

[0037] Obtain environmental data collected by various sensing devices deployed in the environment where the device is located;

[0038] Based on the environmental data, generating behavior description information for the device, and sending the behavior description information to the device;

[0039] During the movement of the device, whether emergency control of the device is required is determined according to the acquired environmental data, and when it is determined that emergency control of the device is required, an emergency control instruction is issued to the device.

[0040] In an optional implementation, the acquiring of environmental data collected by various sensing devices deployed in the environment where the device is located includes:

[0041] Acquire scene data collected by each sensing device deployed in the environment where the device is located, and use the collection of scene data collected by each sensing device as the environment data;

[0042] The generating behavior description information for the device based on the environment data includes:

[0043] Determine a single perception device detection result based on the scene data collected by each perception device; the single perception device detection result indicates the result of tracking and trajectory prediction of a mobile object in the environment where the device is located using the scene data collected by the single perception device;

[0044] The detection results of the single sensing devices corresponding to the sensing devices are integrated to determine the detection results in the environment where the devices are located; the detection results indicate the results of tracking and trajectory prediction of each moving object in the environment where the devices are located;

[0045] Based on the detection result and static environment data of the environment where the device is located, behavior description information is generated for the device.

[0046] In an optional implementation, the acquiring of environmental data collected by various sensing devices deployed in the environment where the device is located includes:

[0047] Acquire scene data collected by each sensing device deployed in the environment where the device is located, perform feature fusion on the scene data collected by each sensing device to obtain fused bird's-eye view features, and use the bird's-eye view features as the environment data;

[0048] The generating behavior description information for the device based on the environment data includes:

[0049] Determining, based on the bird's-eye view feature, a detection result within the environment where the device is located; the detection result indicates a result of tracking and trajectory prediction of each moving object within the environment where the device is located;

[0050] Based on the detection result and static environment data of the environment where the device is located, behavior description information is generated for the device.

[0051] In an optional implementation, during the movement of the device, determining whether emergency control of the device is required according to the acquired environmental data, and when it is determined that emergency control of the device is required, issuing an emergency control instruction to the device includes:

[0052] During the movement of the device, the acquired environmental data is input into an anomaly detection model to predict whether there is an abnormal situation that affects the movement of the device, and the abnormal type and abnormal location corresponding to the abnormal situation;

[0053] When an abnormal situation is predicted, an emergency control instruction is generated according to the abnormal type and abnormal location, and the emergency control instruction is sent to the device.

[0054] In an optional implementation, generating behavior description information for the device based on the environment data includes:

[0055] Based on the environmental data and the perception data uploaded by each other device in the environment where the device is located, behavior description information is generated for the device.

[0056] In a third aspect, an embodiment of the present disclosure further provides a vehicle, comprising a controller, the controller comprising: a memory for storing machine-readable instructions; a processor for calling and executing the machine-readable instructions stored in the memory to implement the above-mentioned first aspect, or steps in any possible implementation of the first aspect.

[0057] In a fourth aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect, are executed, or the above-mentioned second aspect, or any possible implementation of the second aspect, are executed.

[0058] In a fifth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned first aspect, or any possible implementation of the first aspect, or executes the above-mentioned second aspect, or any possible implementation of the second aspect.

[0059] The device movement control method provided by the embodiment of the present disclosure, on the one hand, a device that can move autonomously (such as an autonomous driving vehicle) can achieve autonomous control according to the behavior description information sent from the cloud and the surrounding environment information perceived by itself; on the other hand, during the movement of the device, the device can respond to the emergency control instructions sent from the cloud, realize scene perception and monitoring based on the cloud, and make behavioral decisions and plans for abnormal situations. The present disclosure improves the ability of the device to operate normally in its environment by combining autonomous control of the device with remote braking from the cloud; in addition, the behavior description information and emergency control instructions are generated by the cloud based on the environmental data collected by various sensing devices deployed in the environment where the device is located. In this way, a control method of coordinated cooperation between the device environment, the device, and the cloud is formed, so that during the movement of the device in its environment, the device can respond to the occurrence of abnormal situations in a timely manner, thereby ensuring the normal operation of the device in its environment.

[0060] For a description of the effects of the above-mentioned vehicle, computer equipment, and computer-readable storage medium, please refer to the description of the above-mentioned device movement control method, which will not be repeated here.

[0061] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0063] Figure 1 A flow chart of a device movement control method provided by an embodiment of the present disclosure is shown;

[0064] Figure 2 A flow chart showing another device movement control method provided by an embodiment of the present disclosure;

[0065] Figure 3 A flowchart of another device movement control method provided by an embodiment of the present disclosure is shown;

[0066] Figure 4 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0068] Research has found that some autonomous mobile devices need to be tested in various scenarios after they are offline. However, due to the complex environment of some test scenarios, the autonomous mobility of the device usually cannot meet the requirements for device mobility security and flexibility in these complex environments.

[0069] For example, the driving level supported by the sensors in autonomous vehicles and the domain control equipment in the vehicles is usually L2 Driving Assistance or L2+ Driving Assistance, which does not reach the L4 Driving Assistance required by factory test scenarios. In addition, the factory environment is quite different from other environments, and current autonomous vehicles cannot cope well with abnormal situations that occur when operating in the factory.

[0070] Based on this, the present disclosure provides a device movement control method. On the one hand, a device that can move autonomously (such as an autonomous driving vehicle) can achieve autonomous control according to the behavior description information sent from the cloud and the surrounding environment information perceived by itself; on the other hand, during the movement of the device, the device can respond to the emergency control instructions sent from the cloud, and realize scene perception and monitoring based on the cloud, and make behavioral decisions and plans for abnormal situations. The present disclosure improves the ability of the device to operate normally in its environment by combining autonomous control of the device with remote braking from the cloud; in addition, the behavior description information and emergency control instructions are generated by the cloud based on the environmental data collected by various sensing devices deployed in the environment where the device is located. In this way, a control method of coordinated cooperation between the device environment, the device, and the cloud is formed, so that during the movement of the device in its environment, the device can respond to the occurrence of abnormal situations in a timely manner, thereby ensuring the normal operation of the device in its environment.

[0071] The defects existing in the above schemes and the proposed solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0072] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0073] To facilitate understanding of this embodiment, a device movement control method disclosed in the embodiment of the present disclosure is first introduced in detail. The executor of the device movement control method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities.

[0074] The device movement control method provided by the embodiments of the present disclosure can be applied to scenarios where a device capable of autonomous movement, the environment in which the device is located, and a cloud server coordinately control the movement of the device in its environment. For example, it can be applied to scenarios where a device capable of autonomous movement (such as an autonomous driving vehicle), a factory, and a cloud server coordinately control the movement of the device in a factory environment (such as a factory testing scenario). The above-mentioned devices may include any device capable of autonomous movement, such as vehicles, logistics robots, robot dogs, sweeping robots, etc. In the following embodiments, the device is mainly introduced as a vehicle with autonomous driving capabilities and capable of autonomous movement.

[0075] See also Figure 1 As shown, it is a flow chart of a device movement control method provided by an embodiment of the present disclosure. The method is applied to a device capable of autonomous movement, such as a vehicle, and may specifically include S101 to S103, wherein:

[0076] S101: Obtaining behavior description information sent from the cloud; the behavior description information is used to describe information of at least one target location that the device needs to reach;

[0077] S102: Based on the behavior description information and the surrounding environment information perceived by the device, control the device to move to the at least one target location;

[0078] S103: During the movement of the device, an emergency control instruction issued by the cloud is received, and the emergency control instruction is responded to; the behavior description information and the emergency control instruction are generated by the cloud based on environmental data collected by various sensing devices deployed in the environment where the device is located.

[0079] In the embodiment of the present disclosure, a sensing device and a processor are deployed in the above-mentioned device.

[0080] The above-mentioned sensing devices may include, for example, laser radars, cameras, millimeter wave radars, and ultrasonic radars. In the embodiments of the present disclosure, there may be no specific restrictions on the number and type of sensing devices deployed in the device. The sensing devices deployed in the device can sense the surrounding environment information.

[0081] The processor can generate movement instructions based on the surrounding environment information perceived by the sensing device to achieve autonomous movement in the environment. The processor can also receive emergency control instructions issued by the cloud server and brake the device. The processor can also receive other instructions from the cloud server, such as forward, backward, pull over, turn the left wheel to a preset angle and then move forward, turn the right wheel to a preset angle and then move forward, stop and wait, etc.

[0082] The environment where the above-mentioned devices are located is also equipped with sensing devices, such as conventional high-definition cameras, binocular cameras with depth information, infrared cameras, laser radars, millimeter-wave radars and other sensor devices. In the embodiments of the present disclosure, there is no specific restriction on the number and type of sensing devices deployed in the environment where the devices are located.

[0083] The above-mentioned sensing devices can collect environmental data in the environment where the devices are located and upload the collected environmental data to the cloud. The environmental data can include the movement data of each device, obstacle information in the environment where the devices are located, etc. The environmental data can be used in the cloud to monitor the movement status of each device and perform emergency braking on the device.

[0084] The cloud can receive environmental data uploaded by the sensing devices in the environment where the device is located, and can also receive mobile data uploaded by the device. Based on the mobile data uploaded by the device and the environmental data uploaded by the sensing devices in the environment where the device is located, the cloud can monitor the mobile status of each device, the quality status of the device, etc. The cloud can also generate information describing at least one target location that the device needs to reach based on the environmental data uploaded by the sensing devices in the environment where the device is located, and can also generate emergency control instructions for braking the device, and can also generate other instructions for controlling the device.

[0085] In the disclosed embodiment, the behavior description information may include at least one of the following information: location identification information of at least one target location that the device needs to reach, the location, description information of the behavior that needs to be performed to reach the target location, description information of the target behavior that needs to be performed after reaching the target location, etc. The location identification information may include a workstation identification, a test link identification, a location identification for barge transport into storage, etc. in the environment where the device is located. The location may be location information in a world coordinate system, which may be specifically expressed in latitude and longitude, or in length units.

[0086] In one embodiment, the behavior description information is described in natural language. Based on the behavior description information and the surrounding environment information perceived by the device, controlling the device to move to at least one target location includes: understanding the behavior description information in natural language and parsing the behavior description information into executable control instructions; and controlling the device to move according to the control instructions and the surrounding environment information perceived by the device.

[0087] In the above implementation, the description information of the behavior that needs to be performed to reach the target location can be described in natural language. For example, when the speed limit is XX kilometers per hour, go straight for about XX meters, and there is a YY object in the right front at a distance of XX meters, stop moving forward, and autonomous obstacle avoidance is not allowed during the movement; when the speed limit is XX kilometers per hour, go straight for about XX meters, when the gesture detection of the operator in the right front is a predefined stop gesture, stop in XX seconds, and autonomous obstacle avoidance is not allowed during the movement. If you encounter an obstacle, please stop and wait and remind. If the waiting time exceeds XX seconds, send a notification message to the cloud; when the speed limit is XX kilometers per hour, go straight for about XX meters, and when the voice recognition result of the operator in the right front is stop, stop in XX seconds. It should be understood here that XX is a specific value set according to the actual situation.

[0088] The description information of the target behavior to be performed after reaching the target location can also be described in natural language. For example, stop for XX seconds and wait for the trigger to run; or, please operate according to the operator's voice requirements.

[0089] After obtaining the behavior description information sent from the cloud, the device needs to parse the behavior description information. Here, the traditional natural language understanding method can be used, that is, the behavior description information is parsed and processed according to a predefined set of rules, or the behavior description information can be parsed based on a large language model. In the embodiments of the present disclosure, the method for natural language understanding of the behavior description information may not be specifically limited.

[0090] By understanding the behavior description information in natural language, executable control instructions can be obtained.

[0091] In the above implementation, by using natural language to describe the behavior description information, it is not only convenient to parse the behavior description information, but also convenient to subsequently expand and modify the behavior description information. That is, there is no need to agree on the instruction name on the cloud side and the device side every time the behavior description information is added or modified.

[0092] In one embodiment, when the behavior description information includes description information of a target behavior to be performed after arriving at the target location, the method further includes: after arriving at the target location, performing the target behavior based on the description information of the target behavior. Here, the target behavior is related to the requirements corresponding to the target location.

[0093] The device parses the behavior description information and can determine the target behavior that needs to be performed after arriving at the target location included in the behavior description information. After arriving at the target location, the target behavior corresponding to the target location can be performed. The device can parse the above behavior description information before the device arrives at the target location, or it can occur after the device arrives at the target location.

[0094] Through the above implementation, it is possible to realize that after the device reaches the target location, it can execute the target behavior in an autonomously controlled manner.

[0095] In the disclosed embodiment, the device may obtain the behavior description information sent from the cloud by downloading. In one implementation, the device may download a navigation map from the cloud. The navigation map may include the behavior description information.

[0096] In one implementation, the behavior description information in the navigation map may be multiple parallel lines. For example, from point A to point B on a road section, it may include outdoor scenes, test scenes, maintenance scenes, etc. The behavior description information in the navigation map may include movement strategies in different scenes. During the process of surrounding environment perception, path planning and movement control, the device may select corresponding movement strategies according to different perception scenes.

[0097] In one implementation, the behavior description information may include movement strategies corresponding to a variety of perception scenarios; different perception scenarios correspond to different movement strategies.

[0098] That is, based on the behavior description information and the surrounding environment information perceived by the device, the device is controlled to move to at least one target location, including: based on the surrounding environment information perceived by the device, determining the current target perception scene; the target perception scene belongs to one of multiple perception scenes; and controlling the device movement according to the target movement strategy for the target perception scene indicated by the behavior description information.

[0099] In the above implementation, the type of the perception scene may be determined based on obstacle information in the environment where the device is located. There may be obstacles in the environment where the device is located, and the obstacles may include different types, such as dynamic obstacles, static obstacles, etc. The type of the perception scene may be determined based on the type of the obstacle. For example, when the obstacle is a dynamic obstacle, the type of the perception scene may be a dynamic obstacle perception scene; when the obstacle is a static obstacle, the type of the perception scene may be a static obstacle perception scene.

[0100] The device can determine the target perception scene according to whether the perceived surrounding environment information contains obstacles and the types of obstacles. The device can parse the behavior description information and then determine the target movement strategy corresponding to the type of the target perception scene.

[0101] For example, when the target perception scene is determined to be a dynamic obstacle perception scene, the target movement strategy corresponding to the type of the perception scene can be determined as follows: when there is a dynamic obstacle at a distance of XX meters ahead, stop moving forward, and autonomous obstacle avoidance is not allowed during movement.

[0102] For example, when the target perception scene is determined to be a static obstacle perception scene, the target movement strategy corresponding to the type of perception scene can be determined as follows: stop and wait and be reminded when encountering an obstacle. If the waiting time exceeds XX seconds, a notification message will be sent to the cloud.

[0103] The type of the perception scene may also be determined according to the environment type of the environment in which the device is located. The environment type may include, for example, an outdoor scene, a test scene, etc. Exemplarily, when the environment type is an outdoor scene, the type of the perception scene may be an outdoor perception scene; when the environment type is a test scene, the type of the perception scene may be a test perception scene.

[0104] The device can determine the target environment type of the device's environment based on the perceived surrounding environment information, and then determine the target perception scene corresponding to the target environment type. The device can parse the behavior description information and then determine the target movement strategy corresponding to the type of the target perception scene.

[0105] For example, when the target perception scene is determined to be an outdoor perception scene, the target movement strategy corresponding to the type of perception scene can be determined as follows: when the speed limit is XX kilometers per hour, move at a speed of XX kilometers per hour, autonomous obstacle avoidance is not allowed during movement, if an obstacle is encountered, please stop and wait and be reminded, and if the waiting time exceeds XX seconds, a notification message will be sent to the cloud.

[0106] For another example, when the target perception scenario is determined to be a test scenario, the target movement strategy corresponding to the type of the perception scenario can be determined as: under a speed limit of XX kilometers per hour, go straight for about XX meters, and when the gesture of the operator in front of the right is detected as a predefined stop gesture, stop in XX seconds; when the gesture of the operator in front of the right is detected as a predefined start gesture, start in XX seconds and continue moving.

[0107] The type of perception scene may also be determined based on other characteristics of the environment in which the device is located, which is not specifically limited here.

[0108] Through the above implementation, the device can select corresponding mobile strategies according to different perception scenarios during the process of surrounding environment perception, path planning and mobile control to achieve autonomous control of the device.

[0109] In one embodiment, the method further includes: after the device arrives at the target location, executing an operation instruction of a staff member at the target location; wherein the operation instruction is at least one of a voice instruction, a gesture instruction, or a remote control instruction issued by a remote control device.

[0110] After the device reaches the target location, the staff's gesture recognition or voice recognition can be enabled based on the behavior description information.

[0111] For example, the behavior description information includes: stopping or starting the vehicle according to the operator's gesture, and the operator's gesture recognition can be turned on. After the operator's gesture recognition is turned on, the device can execute the received operator's gesture command.

[0112] For another example, the behavior description information includes: stopping or starting the vehicle according to the voice recognition result, at this time, the staff's voice recognition can be turned on. After the staff's language recognition is turned on, the device can execute the received staff's language instructions.

[0113] After the device arrives at the target location, the device can also receive and execute remote control instructions issued by the staff at the target location using a remote control device.

[0114] Through the above implementation, after the device reaches the target location, it is possible to perform corresponding actions in a combination of human control and autonomous control.

[0115] In other embodiments, when the operation instruction is a voice instruction, after receiving the voice instruction, the target voiceprint information of the voice instruction can be identified; based on the target voiceprint information and the pre-stored voiceprint information of the staff at each operation station, it is determined whether the voice instruction comes from the staff at the station corresponding to the target location; if so, the voice instruction is executed.

[0116] In a specific implementation, the following voice commands may be defined: please stop the vehicle, please start leaving the workstation, please slowly move forward XX meters, please slowly move backward XX meters, please adjust the angle XX degrees, etc.

[0117] In other embodiments, when the operation instruction is a gesture instruction, the recognized target gesture type is determined; the target gesture type belongs to one of a plurality of predefined gesture types, and different gesture types correspond to different gesture instructions; and the gesture instruction corresponding to the target gesture type is executed.

[0118] In a specific implementation, gesture instructions can be defined according to actual needs.

[0119] In further embodiments, different types of operation instructions may be used in combination, for example, simple operations may be performed using gesture instructions, and complex operations may be performed using voice instructions.

[0120] In specific implementations, if the staff's operating instructions exceed the predefined instruction scope, a reminder message can be sent to the staff or the device can report to the cloud.

[0121] Through the above implementation, the safety of the equipment during human control is improved to a certain extent.

[0122] The device may encounter obstacles during movement. In the case of encountering obstacles, the device can take corresponding obstacle avoidance measures in combination with the remote braking method of the cloud, or the device can also avoid obstacles autonomously.

[0123] In case of encountering obstacles, you can combine the remote braking method of the cloud to take corresponding obstacle avoidance measures for the device, specifically:

[0124] In one embodiment, when it is determined based on surrounding environment information that there is an obstacle in the direction of movement of the device, the device is controlled to stop moving and wait; in response to the obstacle not being removed after a set waiting time, an obstacle reminder message is sent to the cloud; and a first instruction issued by the cloud based on the obstacle reminder message is received; the first instruction is used to control the device to continue waiting until the obstacle is removed, or to control the device to move according to a route re-planned by the cloud, or to control the device to move to a target location re-planned by the cloud.

[0125] In the above-mentioned embodiment, when there is an obstacle in the moving direction of the device, the device does not perform autonomous obstacle avoidance but stops and waits.

[0126] Here you can set the waiting time. If the obstacle is not removed after the set waiting time, the device can send an obstacle reminder message to the cloud. The obstacle reminder message can include obstacle information. The obstacle reminder message can be used to remind the cloud to send an obstacle avoidance instruction to the device.

[0127] The first instruction issued by the cloud based on the obstacle reminder information can be used to control the device to continue waiting until the obstacle is removed. After receiving the first instruction, the device can continue to wait until the obstacle is removed.

[0128] Alternatively, the first instruction issued by the cloud based on the obstacle reminder information is used to control the device to move along the route replanned by the cloud. In this case, the first instruction may include other routes replanned by the cloud. After receiving the first instruction, the device will move along the other routes replanned by the cloud.

[0129] Alternatively, the first instruction issued by the cloud based on the obstacle reminder information is used to control the device to move to the target location re-planned by the cloud. In this case, the first instruction may include the target location re-planned by the cloud. After receiving the first instruction, the device will move to the target location according to the target location re-planned by the cloud.

[0130] In one embodiment, the behavior description information may include an arrival order of the target locations. In the case where the first instruction is used to control the device to move to the target location re-planned in the cloud, the method further includes: after the device arrives at the target location re-planned in the cloud, controlling the device to arrive at the remaining unreached target locations in sequence according to the arrival order of the multiple target locations included in the behavior description information; or, after the device arrives at the target location re-planned in the cloud, controlling the device to arrive at the remaining unreached target locations in sequence according to the re-planned arrival order indicated by the first instruction.

[0131] In the above-mentioned implementation, under normal circumstances, the device can reach each target location in the order of arrival of the target location included in the behavior description information. In the case of a target location re-planned in the cloud, after the device reaches the target location re-planned in the cloud, the original order of arrival of each target location is disrupted. In this case, in one way, the device can be controlled to arrive at the remaining unreached target locations in sequence according to the order of arrival of the multiple target locations included in the behavior description information. That is, for the remaining unreached target locations, the device can be controlled to arrive in sequence according to the order of arrival of the multiple target locations included in the behavior description information. In one way, the device can be controlled to arrive at the remaining unreached target locations in sequence according to the re-planned arrival order indicated by the first instruction. That is, the first instruction can include the re-planned arrival order of the cloud, and the device can no longer move according to the original arrival order of each target location, but can arrive at the remaining unreached target locations in sequence according to the re-planned arrival order indicated by the first instruction.

[0132] Through the above implementation, when the device encounters an obstacle, the device is controlled to take corresponding obstacle avoidance measures through remote braking in the cloud, thereby improving the ability of the device to operate normally in its environment.

[0133] In case of encountering obstacles, the device can avoid them autonomously. Specifically:

[0134] In one embodiment, during the movement of the device, the method also includes: when it is determined based on the surrounding environment information that there is an obstacle in the moving direction of the device, controlling the device to autonomously avoid the obstacle; if the device re-plans the moving route based on the obstacle avoidance requirement, sending the re-planned moving route to the cloud; after receiving confirmation information from the cloud regarding the re-planned moving route, controlling the device to move according to the re-planned moving route.

[0135] In the above embodiment, when there is an obstacle in the moving direction, the device can re-plan the moving route, and the re-planned moving route can be sent to the cloud. After confirmation in the cloud, the device can be controlled to move according to the re-planned moving route.

[0136] Through the above implementation, when the device encounters an obstacle, the device re-plans the moving route, thereby ensuring the normal operation of the device in its environment under autonomous control.

[0137] See also Figure 2 FIG. 2 is a flow chart of another device movement control method provided by an embodiment of the present disclosure. The method is applied to a cloud server and may specifically include S201 to S203, wherein:

[0138] S201: Obtain environmental data collected by various sensing devices deployed in the environment where the device is located.

[0139] S202: Generate behavior description information for the device based on the environment data, and send the behavior description information to the device.

[0140] S203: During the movement of the device, determining whether emergency control of the device is required based on the acquired environmental data, and issuing an emergency control instruction to the device when it is determined that emergency control of the device is required.

[0141] In the embodiments of the present disclosure, each sensing device deployed in the environment where the device is located can collect full-scene environmental data in the environment where the device is located in real time.

[0142] Each sensor device deployed in the environment where the device is located can send the collected environmental data to the cloud. When the cloud server obtains environmental data, it can also obtain the internal and external parameter information of each sensor device.

[0143] After the cloud server obtains the environmental data collected by each sensing device, it can then generate behavior description information for the device based on the environmental data.

[0144] In the embodiments of the present disclosure, the device can be first detected and tracked based on the scene data collected by a single perception device to obtain a detection result, and then the detection result corresponding to the single perception device can be fused, and finally, based on the fused detection result, behavior description information can be generated for the device; or the scene data collected by each perception device can be fused to obtain a fusion feature, and then the device can be detected and tracked based on the fusion feature to obtain a detection result, and finally, based on the obtained detection result, behavior description information can be generated for the device.

[0145] Specifically, in one embodiment, environmental data collected by each perception device deployed in the environment where the device is located is obtained, including: obtaining scene data collected by each perception device deployed in the environment where the device is located, and taking the collection of scene data collected by each perception device as environmental data.

[0146] Based on the environmental data, behavior description information is generated for the device, including: determining the detection result of a single perception device based on the scene data collected by each perception device; the detection result of a single perception device indicates the result of tracking and trajectory prediction of mobile objects in the environment where the device is located using the scene data collected by the single perception device; the detection results of the single perception devices corresponding to each perception device are integrated to determine the detection result in the environment where the device is located; the detection result indicates the result of tracking and trajectory prediction of each mobile object in the environment where the device is located; and generating behavior description information for the device based on the detection result and the static environmental data of the environment where the device is located.

[0147] That is, in the above implementation, the scene data collected by a single sensing device can be used to track and predict the trajectory of the moving objects in the environment where the device is located, and obtain the detection result of the single sensing device. Then, the detection results of the single sensing devices corresponding to each sensing device are fused to obtain the fused detection result, that is, the detection result in the environment where the device is located.

[0148] The moving objects may include the device, other devices, and other dynamic objects (such as staff), etc. For example, taking the device as a vehicle, the moving objects may include the current vehicle, other vehicles, pedestrians, etc.

[0149] In one embodiment, environmental data collected by each perception device deployed in the environment where the device is located is obtained, including: obtaining scene data collected by each perception device deployed in the environment where the device is located, performing feature fusion on the scene data collected by each perception device to obtain fused bird's-eye view features, and using the bird's-eye view features as environmental data.

[0150] Based on the environmental data, behavior description information is generated for the device, including: based on the bird's-eye view features, determining the detection results in the environment where the device is located; the detection results indicate the results of tracking and trajectory prediction of each mobile object in the environment where the device is located; based on the detection results and the static environmental data of the environment where the device is located, behavior description information is generated for the device.

[0151] That is, in the above implementation, the scene data collected by each sensing device can be feature fused to obtain the fused bird's-eye view features. Then, the bird's-eye view features are detected to obtain the results of tracking and trajectory prediction of each moving object in the environment where the device is located.

[0152] In a specific implementation, feature extraction can be performed on the scene data collected by each sensing device to obtain features corresponding to each sensing device, and then feature fusion can be performed on the features corresponding to each sensing device to obtain fused bird's-eye view features.

[0153] The cross-attention mechanism can be used to associate the bird's-eye view features with the camera image features, lidar features, or millimeter-wave radar features under the perspective view (PV) to obtain the full-scene bird's-eye view features to be decoded. After decoding the full-scene bird's-eye view features to be decoded, the tracking and trajectory prediction results of each moving object in the environment where the device is located can be obtained.

[0154] In the above two embodiments, the detection results in the environment where the device is located may include the results of tracking and trajectory prediction of each mobile object in the environment where the device is located, and specifically may include the tracking results and trajectory prediction results of each mobile object. The tracking results of each mobile object can be used to determine the speed, acceleration and other information of each mobile object at each moment. Combining the tracking results and trajectory prediction results of each mobile object, a trajectory route that avoids collision can be generated.

[0155] The trajectory prediction results of each moving object can be used to perform a collision test to determine whether the trajectories of each moving object overlap or intersect.

[0156] When the predicted trajectories of various moving objects collide, the cloud server can issue an early warning. At the same time, it can determine whether there are abnormal trajectory sections in the trajectories of various devices based on the predicted trajectories of various moving objects. Abnormal trajectory sections can be overlapping trajectory sections or crossing trajectory sections.

[0157] If there is, the abnormal trajectory section can be adjusted to obtain the adjusted trajectory corresponding to each device. And according to the predicted trajectory of each moving object, it is judged whether there is still an abnormal trajectory section in the adjusted trajectory of each device. If there is still, an emergency braking command can be issued to the target device with the highest speed or the longest abnormal trajectory section among the devices that collided.

[0158] In one implementation, behavior description information may also be generated for the device based on the perception data uploaded by other devices. Generating behavior description information for the device based on environment data includes: generating behavior description information for the device based on environment data and perception data uploaded by other devices in the environment where the device is located.

[0159] The perception data uploaded by other devices refers to the environmental data collected by other devices through their own perception devices.

[0160] In the embodiments of the present disclosure, an abnormality detection model may be used to monitor whether there are abnormal conditions in the environment where the device is located that affect the movement of the device, and then determine whether emergency control of the device is required.

[0161] Through the above implementation, more accurate behavior description information can be generated by combining the environmental data collected by each perception device deployed in the environment where the device is located, and the perception data uploaded by each other device located in the environment where the device is located.

[0162] In one embodiment, during the movement of the device, whether emergency control of the device is required is determined based on the acquired environmental data, and when it is determined that emergency control of the device is required, an emergency control instruction is issued to the device, including: during the movement of the device, the acquired environmental data is input into an anomaly detection model to predict whether there is an abnormal situation that affects the movement of the device, and the abnormal type and abnormal location corresponding to the existing abnormal situation; when it is predicted that an abnormal situation exists, an emergency control instruction is generated based on the abnormal type and abnormal location, and the emergency control instruction is issued to the device.

[0163] In the above implementation, the environmental data containing abnormal conditions collected by the sensing device in the device and the environmental data containing abnormal conditions collected by the sensing device deployed in the environment where the device is located can be used to train the abnormality detection model. The trained abnormality detection model can be used to detect abnormal conditions of the mobile device in the environment where the device is located, and determine the abnormality type and abnormal location corresponding to the existing abnormal conditions.

[0164] Among them, the abnormal type corresponding to the abnormal situation may include equipment congestion, impending collision, etc.

[0165] The emergency control instruction generated according to the abnormal type and abnormal position may include instructions for controlling the equipment to speed up, stop and wait, take a detour, etc. The specific content of the emergency control instruction may not be specifically limited here.

[0166] Through the above implementation mode, by using the abnormal detection model to detect abnormal situations of the equipment, more accurate abnormality types and abnormal locations can be obtained, so that more accurate emergency control instructions can be generated in time for abnormal situations. In this way, when the equipment moves in its environment, the equipment can respond to the occurrence of abnormal situations in time, thereby ensuring the normal operation of the equipment in its environment.

[0167] In the disclosed embodiment, the cloud server may send emergency control instructions to the device in a natural language.

[0168] In one implementation, the cloud server may obtain the mobile control information sent by each device. The cloud server may determine the mobile status information of each device based on the mobile control information sent by each device and the environmental data collected by each sensing device in the environment where the device is located. The cloud server may compare the mobile control information and mobile status information of each device to determine the control capability information of each device. Based on the control capability information of each device, abnormal devices may be determined.

[0169] The above-mentioned mobile control information may include information such as the planned path of the device, mobile strategy, etc. The mobile control information may be determined by the device based on the behavior description information and the perceived surrounding environment information.

[0170] The movement status information may include the movement trajectory and acceleration / deceleration information of the device.

[0171] The control capability information may include the response speed and the fluctuation of acceleration and deceleration. The control capability information is used to characterize the automatic control implementation capability of the equipment.

[0172] The environmental perception capability and automatic control implementation capability of the equipment can be specifically achieved by combining an end-to-end perception model with traditional planning and control (Planning and Control, PNC), or by combining a perception model with a PNC model, or by combining an end-to-end planning model with a vehicle control method, or by combining an end-to-end model with a large visual language model. No specific limitation is made here.

[0173] In one embodiment, the cloud server can obtain the detection results of each detection station on the device and the status information of the device at each detection station, and determine the abnormal device according to the detection results of each detection station on the device and the status information of the device at each detection station.

[0174] The inspection results of the equipment at each inspection station are used to characterize the quality status information of the equipment. The status information of the equipment at each inspection station, such as gear information, posture information, noise information, abnormal sound, sudden acceleration information and emergency braking information, etc.

[0175] Based on the test results of each test station on the equipment and the status information of the equipment at each test station, the mean and variance of the test results corresponding to each test station can be determined. Among them, the difference between the mean and the predefined mean provided by each test station can reflect the deviation of each test result. The variance can also reflect the quality consistency and stability of the equipment. According to the mean and variance of the test results corresponding to each test station, abnormal equipment can be determined.

[0176] A flowchart of another device movement control method provided by an embodiment of the present disclosure is as follows: Figure 3The device movement control method provided by the embodiment of the present disclosure is applied to a factory test scenario in which a vehicle, a factory, and a cloud server coordinately control the operation of a vehicle in a factory environment.

[0177] In the disclosed embodiment, after the vehicle obtains the behavior description information sent from the cloud, it can control the vehicle to travel to at least one target location based on the behavior description information and the surrounding environment information perceived by the vehicle. The factory can upload the environmental data collected by various sensing devices deployed in the factory to the cloud. The cloud can generate behavior description information and emergency control instructions based on the environmental data collected by various sensing devices in the factory, and send the behavior description information to the vehicle to achieve autonomous control of the vehicle, and during the movement of the vehicle, when it is determined that the vehicle needs to be urgently controlled, send emergency control instructions to the vehicle. At the same time, the vehicle status can be detected based on the surrounding environment data perceived by the vehicle.

[0178] The specific process of the above implementation method can be referred to the previous description and will not be repeated here.

[0179] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0180] An embodiment of the present disclosure also provides a vehicle, which includes: a controller, which may include: a memory for storing machine-readable instructions; and a processor for calling and executing the machine-readable instructions stored in the memory to implement the steps of the above-mentioned device movement control method.

[0181] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 4 As shown, it is a schematic diagram of the structure of a computer device 400 provided in an embodiment of the present disclosure, including a processor 401, a memory 402, and a bus 403. Among them, the memory 402 is used to store execution instructions, including a memory 4021 and an external memory 4022; the memory 4021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 401, and the data exchanged with the external memory 4022 such as a hard disk. The processor 401 exchanges data with the external memory 4022 through the memory 4021. When the computer device 400 is running, the processor 401 communicates with the memory 402 through the bus 403, so that the processor 401 executes the following instructions:

[0182] Obtaining behavior description information sent from the cloud; the behavior description information is used to describe information of at least one target location that the device needs to reach;

[0183] Based on the behavior description information and the surrounding environment information perceived by the device, control the device to move to the at least one target location;

[0184] Among them, during the movement of the device, the emergency control instructions issued by the cloud are received, and the emergency control instructions are responded to; the behavior description information and the emergency control instructions are generated by the cloud based on the environmental data collected by various sensing devices deployed in the environment where the device is located.

[0185] Alternatively, when the computer device 400 is running, the processor 401 communicates with the memory 402 via the bus 403, so that the processor 401 executes the following instructions:

[0186] Obtain environmental data collected by various sensing devices deployed in the environment where the device is located;

[0187] Based on the environmental data, generating behavior description information for the device, and sending the behavior description information to the device;

[0188] During the movement of the device, it is determined whether emergency control of the device is required according to the acquired environmental data, and when it is determined that emergency control of the device is required, an emergency control instruction is issued to the device.

[0189] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the device movement control method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0190] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the device movement control method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0191] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0192] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0195] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0196] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A device movement control method, characterized in that: Applicable to devices capable of autonomous movement, including: Obtaining behavior description information sent from the cloud; the behavior description information is used to describe information of at least one target location that the device needs to reach; Based on the behavior description information and the surrounding environment information perceived by the device, control the device to move to the at least one target location; Among them, during the movement of the device, the emergency control instructions issued by the cloud are received, and the emergency control instructions are responded to; the behavior description information and the emergency control instructions are generated by the cloud based on the environmental data collected by various sensing devices deployed in the environment where the device is located.

2. The method according to claim 1, characterized in that The method further comprises: When it is determined based on the surrounding environment information that there is an obstacle in the moving direction of the device, controlling the device to stop moving and wait; In response to the obstacle not being removed after the set waiting time, sending obstacle reminder information to the cloud; Receive a first instruction issued by the cloud based on the obstacle reminder information; the first instruction is used to control the device to continue waiting until the obstacle is removed, or to control the device to move along the route replanned by the cloud, or to control the device to move to the target location replanned by the cloud.

3. The method according to claim 2, characterized in that The behavior description information includes the arrival order of multiple target locations; In the case where the first instruction is used to control the device to move to the target location re-planned by the cloud, the method further includes: After the device reaches the target location re-planned by the cloud, the device is controlled to sequentially reach the remaining target locations that have not been reached according to the arrival order of the multiple target locations included in the behavior description information; or, After the device arrives at the target location re-planned in the cloud, the device is controlled to arrive at the remaining unreached target locations in sequence according to the re-planned arrival order indicated by the first instruction.

4. The method according to claim 1, characterized in that: During the movement of the device, the method further includes: When it is determined based on the surrounding environment information that there is an obstacle in the moving direction of the device, controlling the device to autonomously avoid the obstacle; If the device re-plans the moving route based on the obstacle avoidance requirement, the re-planned moving route is sent to the cloud; After receiving confirmation information from the cloud regarding the re-planned moving route, the device is controlled to move according to the re-planned moving route.

5. The method according to claim 1, characterized in that: The behavior description information includes movement strategies corresponding to a plurality of perception scenarios; different movement strategies correspond to different perception scenarios; The controlling the device to move to the at least one target location based on the behavior description information and the surrounding environment information perceived by the device includes: Determine a current target perception scene based on the surrounding environment information perceived by the device; the target perception scene belongs to one of the multiple perception scenes; Controlling the movement of the device according to the target movement strategy for the target perception scenario indicated by the behavior description information.

6. The method according to any one of claims 1 to 5, characterized in that: The behavior description information also includes description information of a target behavior that needs to be performed after arriving at the target location; the target behavior is related to the requirements corresponding to the target location; The method further comprises: After arriving at the target location, the target behavior is executed based on the description information of the target behavior.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: After the device arrives at the target location, executing the operation instructions of the staff at the target location; The operation instruction is at least one of a voice instruction, a gesture instruction or a remote control instruction issued by a remote control device.

8. The method according to any one of claims 1 to 5, characterized in that: The behavior description information is described in natural language; The controlling the device to move to the at least one target location based on the behavior description information and the surrounding environment information perceived by the device includes: Performing natural language understanding on the behavior description information, and parsing the behavior description information into executable control instructions; The device is controlled to move according to the control instruction and the surrounding environment information sensed by the device.

9. A device movement control method, characterized in that: Applied to a cloud server, the method comprises: Obtain environmental data collected by various sensing devices deployed in the environment where the device is located; Based on the environmental data, generating behavior description information for the device, and sending the behavior description information to the device; During the movement of the device, it is determined whether emergency control of the device is required according to the acquired environmental data, and when it is determined that emergency control of the device is required, an emergency control instruction is issued to the device.

10. The method according to claim 9, characterized in that The step of obtaining environmental data collected by various sensing devices deployed in the environment where the device is located includes: Acquire scene data collected by each sensing device deployed in the environment where the device is located, and use the collection of scene data collected by each sensing device as the environment data; The generating behavior description information for the device based on the environment data includes: Determine a single perception device detection result based on the scene data collected by each perception device; the single perception device detection result indicates the result of tracking and trajectory prediction of a mobile object in the environment where the device is located using the scene data collected by the single perception device; The detection results of the single sensing devices corresponding to the sensing devices are integrated to determine the detection results in the environment where the devices are located; the detection results indicate the results of tracking and trajectory prediction of each moving object in the environment where the devices are located; Based on the detection result and static environment data of the environment where the device is located, behavior description information is generated for the device.

11. The method according to claim 9, characterized in that The step of obtaining environmental data collected by various sensing devices deployed in the environment where the device is located includes: Acquire scene data collected by various sensing devices deployed in the environment where the device is located, perform feature fusion on the scene data collected by the various sensing devices to obtain fused bird's-eye view features, and use the bird's-eye view features as the environment data; The generating behavior description information for the device based on the environment data includes: Determining, based on the bird's-eye view feature, a detection result within the environment where the device is located; the detection result indicates a result of tracking and trajectory prediction of each moving object within the environment where the device is located; Based on the detection result and static environment data of the environment where the device is located, behavior description information is generated for the device.

12. The method according to claim 9, characterized in that During the movement of the device, determining whether emergency control of the device is required according to the acquired environmental data, and issuing an emergency control instruction to the device when it is determined that emergency control of the device is required, including: During the movement of the device, the acquired environmental data is input into an anomaly detection model to predict whether there is an abnormal situation that affects the movement of the device, and the abnormal type and abnormal location corresponding to the abnormal situation; When an abnormal situation is predicted, an emergency control instruction is generated according to the abnormal type and abnormal location, and the emergency control instruction is sent to the device.

13. The method according to claim 9, characterized in that Based on the environment data, generating behavior description information for the device, including: Based on the environmental data and the perception data uploaded by each other device in the environment where the device is located, behavior description information is generated for the device.

14. A vehicle, characterized in that: include: A controller, the controller comprising: a memory for storing machine-readable instructions; A processor is used to call and execute the machine-readable instructions stored in the memory to implement the steps of the device movement control method as described in any one of claims 1 to 8.

15. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the device movement control method as described in any one of claims 1 to 8 are executed, or the steps of the device movement control method as described in any one of claims 9 to 13 are executed.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the device movement control method as described in any one of claims 1 to 8, or executes the steps of the device movement control method as described in any one of claims 9 to 13.

Images