Following type carrying method and device and storage medium

Abstract

The invention provides a following type carrying method and device and a storage medium. Comprising the following steps: acquiring image information of a following object in advance, and storing the image information of the following object; then obtaining a real-time environment image, and obtaining a target following object based on the stored following object image information and the real-time environment image; and outputting a following control instruction according to the target following object, so that the current carrying device responds to the following control instruction and executes real-time following work on the target following object. The following object is locked by learning the body image information of the following object, the following work is controlled through the image information of the environment, and a more accurate following effect is achieved.

Description

Technical Field

[0001] The present invention relates to a cargo carrying method, and particularly to a following type carrying method, device and storage medium, belonging to the technical field of following type carrying. Background Art

[0002] When people carry heavy objects, it is quite labor-consuming and machines are needed to assist in carrying goods. However, the existing small carts need to be actively pulled by people, which not only consumes physical strength but also causes inconvenience to people during the progress. The existing automatic following carrying devices detect following objects based on radio signals and keep a certain distance from the following objects to achieve the function of automatic following. However, this device cannot learn the information of the following object in advance, and there are problems such as relying on auxiliary devices to locate the following object and the following route being prone to deviation. Summary of the Invention

[0003] In view of this, the present invention provides a following type carrying method, device and storage medium to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0004] The technical solution of the embodiment of the present invention is implemented as follows: A following type carrying method is provided, which pre-obtains following object image information and stores the following object image information; the following type carrying method includes the following steps:

[0005] Obtain a real-time environment image to obtain a target following object based on the stored following object image information and the real-time environment image; output a following control instruction according to the target following object, so that the current carrying device responds to the following control instruction and performs real-time following work on the target following object.

[0006] Further preferably: The following object image information at least includes partial body information or global body information of the following object.

[0007] Further preferably: The storing of the following object image information includes: preprocessing the following object image information and extracting key features of the following object image information to obtain following feature information for storage.

[0008] Further preferably: The obtaining of the target following object includes: identifying the real-time environment image based on an artificial intelligence method to search for the target following object in the real-time environment image in combination with the stored following feature information.

[0009] Further preferably: The outputting of the following control instruction includes: obtaining position information of the target following object based on the target following object and the real-time environment image, and generating a following control instruction based on the position information.

[0010] Further preferably: It further includes: in response to a voice control instruction, generating a target control instruction or regenerating a following control instruction based on the voice control instruction and a real-time environmental image; enabling the current transport device to respond to the target control instruction and execute corresponding moving actions; or enabling the current transport device to respond to the regenerated following control instruction and execute real-time following work on a target following object.

[0011] Further preferably: The generating of the target control instruction or the regenerating of the following control instruction includes: analyzing corresponding semantic information based on the voice control instruction, and generating a target control instruction or regenerating a following control instruction based on the analysis result of the semantic information, the position information, and the real-time environmental image.

[0012] Based on the same concept, the present invention further provides a following transport device, which at least includes a transport device, a motion system, and a following control system; the following control system is used to adopt the above-mentioned following transport method to output a following control instruction; the motion system is installed on the transport device, and the motion system is used to respond to the following control instruction and execute real-time following work on a target following object.

[0013] Further preferably: The following transport device further includes a voice control system; the voice control system is used to adopt the above-mentioned following transport method to output a target control instruction or re-output a following control instruction, enabling the motion system to respond to the target control instruction and execute corresponding moving actions; or enabling the current transport device to respond to the regenerated following control instruction and execute real-time following work on a target following object.

[0014] Based on the same concept, the present invention further provides a storage medium, in which a computer program is stored, and wherein the computer program is set to execute the above-mentioned following transport method when running.

[0015] Due to the above technical solutions adopted in the embodiments of the present invention, it has the following advantages:

[0016] The present invention locks the following object by learning the body image information of the following object, and controls the following work through the image information of the surrounding environment, achieving a more accurate following effect.

[0017] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the above-described illustrative aspects, embodiments, and features, further aspects, embodiments, and features of the present invention will be readily apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a flowchart of the following-type carrying method described in the present invention.

[0020] Figure 2 This is a framework diagram of the following-type carrying device described in the present invention. Detailed implementation manners

[0021] In the following, only some exemplary embodiments are briefly described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present invention. Therefore, the drawings and the description are considered to be exemplary in nature rather than restrictive.

[0022] The following will describe the embodiments of the present invention in detail with reference to the accompanying drawings.

[0023] As Figure 1 shown, the embodiments of the present invention provide a following-type carrying method.

[0024] Pre-obtain the following object image information and store the following object image information; the following-type carrying method includes the following steps:

[0025] Obtain the real-time environment image to obtain the target following object based on the stored following object image information and the real-time environment image; output a following control instruction according to the target following object, so that the current carrying device responds to the following control instruction and performs the real-time following work on the target following object.

[0026] In this embodiment, specifically: the following object image information at least includes the body local information or the body global information of the following object.

[0027] Among them, since the following object image information at least includes the body local information or the body global information of the following object, a clear image including the whole body or key parts (such as the face, upper body, etc.) of the following object can be taken. For example, in order to more accurately identify the following object later, a full-body photo of the following object standing frontally is taken.

[0028] In this embodiment, specifically, the storage of the image information of the following object includes: preprocessing the image information of the following object, and extracting key features from the image information of the following object to obtain following feature information for storage.

[0029] Among them, an image cleaning operation is performed on the image information of the following object obtained by shooting to remove noise and interference information in the image. Common filtering algorithms, such as Gaussian filtering, can be used to process the image to make it smoother and clearer, facilitating subsequent feature extraction.

[0030] Furthermore, key features are extracted from the preprocessed image information of the following object based on a deep learning algorithm. For example, using a convolutional neural network (CNN) model, the preprocessed image of the following object is input into the trained CNN model, and the model will automatically learn and extract unique features of the following object, such as facial features, body contour features, etc. These feature information are the following feature information.

[0031] In this embodiment, specifically, the obtaining of the target following object includes: identifying the real-time environment image based on an artificial intelligence method to search for the target following object in the real-time environment image in combination with the stored following feature information.

[0032] Among them, the camera of the following transportation device continuously shoots the real-time environment image of, such as, a warehouse. Based on an artificial intelligence method, for example, using a pre-trained object detection model (such as the YOLO series) to identify the real-time environment image. The recognition result is compared with the stored following feature information to search for the part in the real-time environment image that matches the stored following feature information. If a region highly matching the facial features of the following object A is found in the image, the person corresponding to this region is determined as the target following object A.

[0033] In this embodiment, specifically, the output of the following control instruction includes: obtaining the position information of the target following object based on the target following object and the real-time environment image, and generating a following control instruction based on the position information.

[0034] Among them, according to the pixel position of the target following object A in the real-time environment image, combined with the parameters and installation position of the camera of the following transportation device, the position information of the target following object A in the actual space of the warehouse is calculated through a computer vision algorithm; based on the position information of the target following object A and the layout of the warehouse in the real-time environment image (such as shelf positions, aisle widths, etc.), a path planning algorithm (such as the A* algorithm) is used to calculate the route for the following transportation device to reach the target following object A. For example, considering the traffic conditions in the warehouse aisles and avoiding congested areas, a shortest and feasible route is planned.

[0035] Further, according to the calculated route, a following strategy is formulated. If the route is to go straight along the main aisle of the warehouse, the following strategy can be to maintain a certain safety distance (such as 2 meters) and follow at a constant speed of 0.3 m / s.

[0036] Further, according to the formulated following strategy, specific following control instructions are generated, such as "move forward at a speed of 0.3 m / s and maintain a distance of 2 meters from the target", and the instructions are sent to the motion system of the following transport device. The operating state is updated every preset time (such as 0.1 second), including speed, direction, and relative distance information. Then, based on the following strategy, the following control instructions are resent. The motion system controls the transport device to perform real-time following of the target following object A according to the instructions.

[0037] In this embodiment, specifically: it further includes: in response to a voice control instruction, generating a target control instruction or regenerating a following control instruction based on the voice control instruction and the real-time environmental image; enabling the current transport device to respond to the target control instruction and perform corresponding moving actions; or enabling the current transport device to respond to the regenerated following control instruction and perform real-time following of the target following object. Among them, generating the target control instruction or regenerating the following control instruction includes: analyzing the corresponding semantic information based on the voice control instruction, and generating a target control instruction or regenerating a following control instruction based on the analysis result of the semantic information, the position information, and the real-time environmental image.

[0038] Among them, during the following process, the transport device can also be controlled by voice. For example, the target following object A issues a voice control instruction: "Quickly come in front of me." The voice control system recognition module receives this instruction. Semantic analysis is performed on the voice control instruction "Quickly come in front of me" to understand that its core requirement is to make the transport device quickly move in front of the target following object A.

[0039] Further, combining the analysis result of the semantic information, the position information of the target following object A, and the real-time environmental image, considering that it is necessary to quickly reach in front of the target following object A, the path planning algorithm is used again to calculate the route. For example, the original following route is to follow behind the target following object A, and now it may be necessary to quickly bypass to the front of the target following object A through the adjacent aisle. According to the regenerated route, a motion control strategy is generated. Such as "at a speed of 0.8 m / s, turn right through the adjacent aisle and quickly move to 1 meter in front of the target".

[0040] Further, according to the motion control strategy, a target control instruction "turn right at a speed of 0.8 m / s and move 1 meter in front of the target" is generated and sent to the motion system of the carrier device, thereby adjusting the motion state of the carrier device. The operating state, including speed, direction, and relative distance information, is updated every preset time (such as 0.1 second), and then a target control instruction is regenerated based on the motion control strategy to achieve real-time movement operations.

[0041] Further, if the voice control instruction at this time is "slow down", then a following control instruction is regenerated based on this instruction. The specific control method is as described in the above embodiments and will not be elaborated here.

[0042] Based on the same concept, as Figure 2 shown, the present invention also provides a following type carrier device, which at least includes a carrier device, a motion system, and a following control system; the following control system is used to adopt the above-mentioned following type carrier method to output a following control instruction; the motion system is installed on the carrier device, and the motion system is used to respond to the following control instruction and perform real-time following work on the target following object.

[0043] Among them, the motion system can be any of the following situations: only having a wheeled motion system, only having a legged motion system, or having both a wheeled and a legged motion system and being able to automatically switch. The wheeled motion system refers to a system that realizes the movement of the device based on wheels, and this system is usually suitable for moving on flat terrains such as indoor corridors and outdoor sidewalks. The legged motion system refers to a system that realizes the movement of the device based on mechanical legs, and this system is usually suitable for moving on rough terrains such as indoor stairs and outdoor mountains. When the following type carrier device has both a wheeled and a legged motion system and can automatically switch, the device can flexibly select the most suitable motion system according to the environmental terrain, so as to achieve following of the target. The carrier device can be any device with a cargo-carrying function. For example, in a travel scenario, the carrier device can be a suitcase. Another example is that in a warehouse handling scenario, the carrier device can be a flatbed truck on which goods can be stacked arbitrarily. Another example is that in a daily moving scenario, the carrier device can be a cargo box. The carrier devices in the above scenarios can automatically follow people.

[0044] In some embodiments, the following type carrier device is configured with a camera. Before the cargo-carrying task is executed, the user inputs face image information and image information of the front, back, left, and right directions of the whole body in front of the camera. After the cargo-carrying task starts, the user, as the following object, can walk freely. The following type carrier device will lock the following object and control the movement on flat terrains by driving the wheels of the device, so as to maintain following of the target at a certain following distance.

[0045] In other embodiments, the follow - type transport device is equipped with a camera. Before the cargo - carrying task is executed, the user inputs face image information and image information of the front, back, left, and right directions of the whole body in front of the camera. After the cargo - carrying task starts, the user, as the following object, can walk freely. The follow - type transport device locks the following object. When the following object walks on flat terrains such as corridors, living rooms, sidewalks, and roads, the follow - type transport device retracts its mechanical feet and uses wheels to control the movement of the device; when the following object walks on rough terrains such as stairs and mountain roads, the follow - type transport device retracts the wheels and uses mechanical feet to control the movement of the device. When the following object crosses different terrain areas, the follow - type transport device automatically switches the motion system suitable for the area and maintains following the target at a certain following distance.

[0046] In this embodiment, specifically: the follow - type transport device further includes a voice control system; the voice control system is used to adopt the above - mentioned follow - type transport method to output a target control instruction or re - output a follow - up control instruction, so that the motion system responds to the target control instruction and executes the corresponding movement action; or enables the current transport device to respond to the re - generated follow - up control instruction and execute the real - time following work of the target following object.

[0047] Among them, the voice control system formulates a motion control strategy according to the voice information of the following object and the image information of the current environment, and outputs an instruction to control the motion system of the device, so as to execute more complex motion instructions on the basis of the following function.

[0048] The input information of the voice control system includes: voice information from the following object. This voice information includes control instructions for the following device, such as "start following", "stop following", "wait at the stairway entrance", "keep the following distance within 3 meters", etc.; or it is the image information of the environment where the following device is located input currently, and this image information can be obtained by the following device through real - time shooting by the camera. The voice control system can analyze the voice information from the following object based on artificial intelligence methods, infer its semantics to the conveyed control instructions. The voice control system can analyze the currently input environment image information based on artificial intelligence methods, combine the recognized voice control instructions, calculate the route to complete the control instructions, and output the current optimal control instruction to control the motion system, so as to realize the execution of the voice instructions.

[0049] In some embodiments, the follow - type transport device is equipped with a camera. Before the cargo - carrying task is executed, the user inputs the face image information and the image information of the front, back, left, and right directions of the whole body in front of the camera. After the cargo - carrying task starts, the user, as the following object, can walk freely. The follow - type transport device will lock the following object and control its movement on any terrain through the mechanical feet of the driving device, so as to maintain following the target at a certain following distance. The user inputs through voice to the following device, telling the device "start following". At this time, after the following device understands the semantics of "start following", it activates the automatic following function. The user inputs through voice to the following device, telling the device "stop following". At this time, after the following device understands the semantics of "stop following", it pauses the automatic following function and waits in place. The user inputs through voice to the following device, telling the device "wait at the stairway entrance". At this time, after the following device understands the semantics of "wait at the stairway entrance", it identifies the stairway entrance in the environment and waits at the stairway entrance. The user inputs through voice to the following device, telling the device "maintain a following distance of 1 meter". At this time, after the following device understands the semantics of "maintain a following distance of 1 meter", it adjusts the movement speed and maintains a following distance of 1 meter from the user within the allowable error range.

[0050] Based on the same concept, the present invention also provides a storage medium, in which a computer program is stored. Among them, the computer program is set to execute the above - mentioned follow - type transport method when running.

[0051] As described above, the above are only the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art in the technical field disclosed by the present invention can easily think of various changes or substitutions within the technical scope disclosed by the present invention, and these should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A follow-up transport method, characterized in that: Pre-acquire image information of a following object and store the image information of the following object; The following type carrying method comprises the following steps: Acquire a real-time environment image to obtain a target following object based on the stored following object image information and the real-time environment image; A following control instruction is output according to the target following object, so that the current carrier responds to the following control instruction and performs real-time following work on the target following object.

2. The following type transport method according to claim 1, characterized in that: Also includes: In response to the voice control instruction, generating a target control instruction or regenerating a follow-up control instruction based on the voice control instruction and the real-time environment image; The current carrier responds to the target control instruction and performs a corresponding movement action; or the current carrier responds to the regenerated following control instruction and performs real-time following work on the target following object.

3. The follow-up transport method according to claim 1, characterized in that: The following object image information includes at least body part information or body global information of the following object.

4. The follow-up transport method according to claim 3, characterized in that: The storing of the image information of the followed object includes: The image information of the following object is preprocessed, and key features are extracted from the image information of the following object to obtain following feature information for storage.

5. The follow-up transport method according to claim 4, characterized in that: The obtaining of the target following object comprises: The real-time environment image is recognized based on an artificial intelligence method to search for a target following object in the real-time environment image in combination with the stored following feature information.

6. The follow-up transport method according to claim 5, characterized in that: The output follows the control instruction, including: The position information of the target following object is acquired based on the target following object and the real-time environment image, so as to generate a following control instruction based on the position information.

7. The following type transport method according to claims 2 and 6, characterized in that: The generating of the target control instruction or the regenerating of the follow-up control instruction comprises: The corresponding semantic information is analyzed based on the voice control instruction, so as to generate a target control instruction or regenerate a follow-up control instruction based on the analysis result of the semantic information, the position information and the real-time environment image.

8. A follow-up transport device, characterized in that: The following type carrier equipment at least comprises a carrier, a motion system and a following control system; The following control system is used to adopt the following transport method as described in any one of claims 1, 3-6 to output following control instructions; The motion system is installed on the carrier, and is used to respond to the following control instruction and perform real-time following work on the target following object.

9. The follow-on vehicle according to claim 8, characterized in that: The following type carrier equipment also includes a voice control system; The voice control system is used to adopt the following transportation method as described in any one of claims 2 and 7 to output the target control instruction and re-output the following control instruction, so that the motion system responds to the target control instruction and performs the corresponding movement action; or makes the current transportation device respond to the regenerated following control instruction to perform real-time following work on the target following object.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the follow-on transportation method according to any one of claims 1 to 7 when running.

Images