A control method and device for unmanned equipment

Abstract

This specification discloses a control method and device for unmanned equipment, which is applied to the field of unmanned driving. First, image data containing a target object collected by the unmanned equipment is obtained. Secondly, the observation position corresponding to the target object is determined based on the image data. Then, according to the lighting information of the environment in which the unmanned equipment is located when collecting image data and the observation position, a tracking path for the unmanned equipment to track the target object is planned. Finally, the unmanned equipment is controlled according to the tracking path. This method can plan the tracking path based on the lighting information of the environment in which the unmanned equipment is located when collecting image data and the observation position, thereby increasing the shooting range of the unmanned equipment for collecting image data, allowing the unmanned equipment to collect image data with higher image quality, and then better identify the target object in the image data, thereby better tracking the target object or accurately docking at the location of the target object.

Description

Technical Field

[0001] This specification relates to the field of unmanned driving technology, and in particular to a control method and device for unmanned equipment. Background Art

[0002] With the continuous development of unmanned driving technology, unmanned equipment such as unmanned vehicles, unmanned controlled robots, and drones have been applied to many fields, bringing great convenience to business execution in these fields.

[0003] While driving, the unmanned vehicle can collect image data through a camera and identify the target object in the image data in order to dock at the target object's location. In the prior art, the image data collected by the unmanned vehicle through the camera may be affected by factors such as strong light and tree shade, resulting in the unmanned vehicle being unable to effectively identify the target object. For example, in the case of strong sunlight, the reflection of sunlight caused by sunlight shining on the target object may cause a large light spot to appear in the collected image data, resulting in image data with poor image quality, making it impossible for the unmanned vehicle to identify the target object in the image data, thereby causing the unmanned vehicle to be unable to track the target object or accurately dock at the target object's location.

[0004] Therefore, how to effectively identify the target object in the image data so that the unmanned equipment can accurately dock at the location of the target object is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a control method and device for unmanned equipment, a computer-readable storage medium, and unmanned equipment to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a control method for unmanned equipment, which is applied to the field of unmanned driving and includes:

[0008] Acquire image data containing target objects collected by unmanned equipment;

[0009] determining an observation position corresponding to the target object based on the image data;

[0010] Planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position;

[0011] The unmanned equipment is controlled according to the tracking path.

[0012] Optionally, planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position specifically includes:

[0013] Determining, based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, the position at which the unmanned device is located when collecting the image data that meets the set conditions as the target point;

[0014] According to the target point, a tracking path for the unmanned equipment to track the target object is planned.

[0015] Optionally, determining, based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, the position at which the unmanned device collects the image data that meets the set conditions as the target point specifically includes:

[0016] According to the lighting information of the environment in which the unmanned equipment is located when collecting the image data and the observation position, the position where the image quality of the image data collected by the unmanned equipment is greater than the set image quality threshold is determined as the target point.

[0017] Optionally, planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position specifically includes:

[0018] determining an observation trajectory including a plurality of observation points for observing the target object based on lighting information of an environment in which the unmanned equipment is located when collecting the image data and the observation position;

[0019] determining, based on image data collected at the plurality of observation points for observing the target object when the unmanned device travels along the observation trajectory, an observation point that meets set conditions from the plurality of observation points for observing the target object as a target point;

[0020] The tracking path is determined according to the target point.

[0021] Optionally, based on image data collected at the plurality of observation points for observing the target object when the unmanned device travels along the observation trajectory, determining an observation point that meets set conditions from the plurality of observation points for observing the target object as the target point specifically includes:

[0022] For each observation point, image data collected by the unmanned equipment at the observation point for the target object is obtained as image data corresponding to the observation point;

[0023] Determining the image quality of the image data corresponding to the observation point;

[0024] If the image quality of the image data corresponding to the observation point is greater than the set image quality threshold, the observation point is determined to be an observation point that meets the set conditions, is used as the target point, and the vehicle stops traveling along the observation trajectory.

[0025] Optionally, based on image data collected at the plurality of observation points for observing the target object when the unmanned device travels along the observation trajectory, determining an observation point that meets set conditions from the plurality of observation points for observing the target object as the target point specifically includes:

[0026] Acquire, according to the observation trajectory, image data collected by the unmanned equipment for the target object at all observation points in the observation trajectory, so as to construct an image set corresponding to the observation trajectory;

[0027] From the image set, the image data with the highest image quality is determined as the optimal image data, and the observation point where the unmanned equipment collects the optimal image data is determined as the target point.

[0028] Optionally, planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position specifically includes:

[0029] determining, based on the lighting information of the environment in which the unmanned equipment was located when collecting the image data and the observation position, a number of historical observation points used by the unmanned equipment to observe the target object;

[0030] A tracking path for the unmanned equipment to track the target object is planned based on the multiple historical observation points.

[0031] Optionally, planning a tracking path for the unmanned device to track the target object based on the plurality of historical observation points specifically includes:

[0032] For each historical observation point, obtain historical image data collected by unmanned equipment at the historical observation point;

[0033] If it is determined that the image quality corresponding to the historical image data is greater than the set historical image quality threshold, the historical observation point is used as a preferred historical observation point;

[0034] Based on the determined preferred historical observation points, a tracking path for the unmanned equipment to track the target object is planned.

[0035] Optionally, planning a tracking path for the unmanned device to track the target object based on the plurality of historical observation points specifically includes:

[0036] Planning a tracking path passing through all the historical observation points based on the plurality of historical observation points;

[0037] Controlling the unmanned equipment according to the tracking path specifically includes:

[0038] After determining that the unmanned device has completed the tracking path, the unmanned device is controlled to move to the next observation area.

[0039] This specification provides a control device for unmanned equipment, which is applied to the field of unmanned driving and includes:

[0040] An acquisition module is used to acquire image data containing target objects collected by unmanned equipment;

[0041] a determination module, configured to determine an observation position corresponding to the target object based on the image data;

[0042] A tracking module, configured to plan a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position;

[0043] A control module is used to control the unmanned equipment according to the tracking path.

[0044] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the control method of the unmanned equipment described above is implemented.

[0045] This specification provides an unmanned device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the control method of the unmanned device described above is implemented.

[0046] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0047] In the unmanned device control method provided in this specification, first, image data containing a target object captured by the unmanned device is acquired. Second, the observation position corresponding to the target object is determined based on the image data. Then, a tracking path for the unmanned device to track the target object is planned based on the lighting information of the environment in which the unmanned device was collecting the image data and the observation position. Finally, the unmanned device is controlled according to the tracking path.

[0048] As can be seen from the above method, this method can plan the tracking path for the unmanned device to track the target object based on the lighting information of the environment and the observation position when the unmanned device collects image data, and control the unmanned device according to the tracking path. Compared with the existing technology, which does not consider that the image data collected by the unmanned device through the camera may be affected by strong light, shade, etc. to perform trajectory planning, this method can plan the tracking path based on the lighting information of the environment and the observation position when the unmanned device collects image data, thereby increasing the shooting range of the unmanned device to collect image data with higher image quality, and then better identify the target object in the image data, thereby better tracking the target object or accurately docking at the target object's location. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0050] Figure 1 This is a flow chart of a control method for an unmanned device in this specification;

[0051] Figure 2 A schematic diagram of an observation trajectory including several observation points provided in this manual;

[0052] Figure 3 A schematic diagram of a control device for unmanned equipment provided in this manual;

[0053] Figure 4 The corresponding Figure 1 Schematic diagram of unmanned equipment. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0055] Figure 1 The following is a flow chart of a control method for an unmanned device in this specification, which specifically includes the following steps:

[0056] S100: Acquire image data containing a target object collected by an unmanned device.

[0057] The execution subject of the control method of the unmanned equipment involved in this specification can be the unmanned equipment or an electronic device such as a server installed on the unmanned equipment. For the sake of convenience of description, the control method of the unmanned equipment provided in this specification is described below using the unmanned equipment as the execution subject.

[0058] In the embodiments of this specification, the unmanned device can obtain image data containing target objects collected by itself. The image data mentioned here can be image data obtained by a camera set on the unmanned device. The target objects mentioned here can include obstacles encountered by the unmanned device during driving, or can refer to markers used to identify the unmanned device stop. Among them, the markers set at the unmanned device stop can be provided with a Digital Object Unique Identifier (DOI) (such as a QR code, barcode, etc.).

[0059] The unmanned equipment mentioned in this specification may refer to autonomous driving devices such as drones, unmanned vehicles, robots, and automatic delivery equipment. Based on this, unmanned equipment that utilizes the unmanned equipment control methods provided in this specification can be used to perform delivery tasks in the delivery field, such as using unmanned equipment for express delivery, logistics, and food delivery.

[0060] S102: Determine an observation position corresponding to the target object according to the image data.

[0061] In the embodiments of this specification, the unmanned device can determine the observation position corresponding to the target object based on the image data. The observation position corresponding to the target object mentioned here can refer to the position of the unmanned device itself when it identifies the target object through the image data. Accordingly, the unmanned device can determine the distance between the target object and the unmanned device through the image data, and determine the position information corresponding to the target object based on the position information of the unmanned device itself and the distance between the target object and the unmanned device, and then track the target object based on the position information corresponding to the target object.

[0062] It should be noted that, when the unmanned device is located at the observation position and is not affected by environmental factors, it can use image data to well identify the information contained in the target object. However, the recognition situation in actual applications is often not ideal. The target object may be affected by factors such as strong light and shade, which may cause the unmanned device to be unable to identify the information contained in the target object from the collected image data corresponding to the target object. Therefore, the unmanned device can identify the approximate shape of the target object through image data, determine that the image data contains the target object to be identified, and thereby determine the observation position. Therefore, the observation position mentioned in this manual can also be understood as the location where the unmanned device first identifies the target object through image data.

[0063] For example, if the target object is a marker with a QR code, the unmanned device can use the image data to identify the approximate shape of the QR code to determine the observation location. For another example, if the target object is a road sign, the unmanned device can use the image data to identify the approximate shape of the road sign to determine the observation location. For another example, if the target object is a vehicle, the unmanned device can use the image data to identify the approximate shape of the vehicle to determine the observation location. Of course, it is also possible to determine the distance between the unmanned device and the target object based on human experience, at which the unmanned device can recognize the information contained in the target object, and then use this distance to determine the observation location.

[0064] In the embodiments of this specification, there are multiple methods for unmanned equipment to recognize image data. For example, if the target object is an identifier with a QR code, the unmanned equipment can decode the QR code in the image data to determine the specific information contained in the QR code. For another example, if the target object is a road sign, the unmanned equipment can find the area with text based on a text recognition algorithm, and then recognize the specific information of the text in the area. For another example, if the target object is a vehicle, the unmanned equipment can find the area where the license plate number on the vehicle is located based on a text recognition algorithm, and then recognize the specific information of the license plate number in the area. This specification does not specifically limit the method for recognizing image data and the text recognition algorithm.

[0065] S104: Planning a tracking path for the unmanned device to track the target object according to the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position.

[0066] S106: Control the unmanned equipment according to the tracking path.

[0067] In actual applications, when unmanned vehicles are driving, the image data collected by the camera may be affected by factors such as strong light and tree shade, which may result in the unmanned device being unable to effectively identify the target object. For example, when sunlight is strong, the reflection of sunlight caused by sunlight shining on the target object may cause a large light spot to appear in the collected image data, making it impossible for the unmanned device to identify the target object in the image data. Since the light spot caused by strong light such as sunlight has the strongest effect only at specific angles, in order to avoid the above situation, the unmanned device can use the camera to shoot the target object at different angles to find the shooting angle that has the least impact on the image quality of the image data. This method improves the image quality of the collected image data, and recognizes the image data corresponding to the shooting angle, thereby increasing the success rate of image recognition.

[0068] In the embodiments of this specification, the unmanned device can plan a tracking path for tracking a target object based on the lighting information of the environment in which it collects image data and the observation position. The lighting information mentioned here can refer to the spot size in the image data. In other words, the unmanned device can determine the tracking path for the image data with the smaller spot size based on the spot size and observation position in each collected image data.

[0069] For example, the unmanned vehicle can compare the spot sizes in the collected image data in real time. If the spot size in the image data collected at the current moment is determined to be larger than the spot size in the image data collected at the previous moment, the unmanned vehicle determines that the tracking path currently used by the unmanned vehicle to track the target object is unreasonable and needs to redefine the tracking path. If the spot size in the image data collected at the current moment is determined to be smaller than the spot size in the image data collected at the previous moment, the unmanned vehicle will continue to follow the tracking path currently used by the unmanned vehicle to track the target object.

[0070] In the embodiments of this specification, the unmanned device can determine the location at which the unmanned device collected image data that met set conditions based on the lighting information of the environment in which it collected the image data and the observation position, and use this location as the target point. The set conditions mentioned here can mean that the light spot in the image data does not affect the unmanned device's ability to recognize the information contained in the image data, or that the light spot is absent in the image data. Based on the target point, the unmanned device then plans a tracking path for tracking the target object. Finally, the unmanned device can control the unmanned device according to the tracking path.

[0071] Specifically, the unmanned device can predict the location of the unmanned device when it collects image data that meets the set conditions based on the changes in the light spot in the image data collected over a period of time, the location of the unmanned device, and the speed of the unmanned device, as the target point.

[0072] Furthermore, the unmanned device can determine, based on the lighting information of the environment in which it is collecting the image data and the observation location, the location at which the image quality of the image data collected by the unmanned device exceeds a set image quality threshold, and use that location as the target point. In other words, the set condition can be that the image quality of the image data collected at the location of the unmanned device exceeds the set image quality threshold.

[0073] It should be noted that since the target object can be a dynamic obstacle, the target point can also refer to the relative positional relationship between the unmanned device and the target object. Based on the relative positional relationship between the unmanned device and the target object, as well as the target object's motion trajectory, the unmanned device can plan a tracking path for the target object.

[0074] In practical applications, while collecting image data, the unmanned vehicle captures multiple images of the target object using its camera at different angles and positions. The system then selects the image data that meets the specified criteria and determines the location of the unmanned vehicle when it captured the image data. Finally, the unmanned vehicle determines a tracking path based on this location.

[0075] In the embodiments of this specification, the unmanned device can determine an observation trajectory including observation points of several observation targets based on the lighting information of the environment in which it is located when collecting image data and the observation position.

[0076] Specifically, the unmanned device can determine a collection range for capturing images of the target object based on the distance between the observation location and the target object's location. Based on the collection range, several observation points for observing the target object are determined. Finally, based on the several observation points for observing the target object, an observation trajectory containing the several observation points for observing the target object is determined.

[0077] Furthermore, the observation trajectory actually refers to the distance from the observation position to the target object as the radius and the target object as the center. Therefore, each trajectory point on the observation trajectory can also be regarded as the observation point of the target object. Figure 2 shown.

[0078] Figure 2 A schematic diagram of an observation trajectory including several observation points provided in this manual.

[0079] exist Figure 2 In the figure, the five-pointed star represents the target object, serving as the center of the circle. The triangle represents the location of the unmanned vehicle. The solid circle represents the observation point. After the unmanned vehicle reaches the observation point, it must move in a 180-degree semicircular motion to cover all possible angles of the target object, ensuring that the unmanned vehicle can capture the target object from different angles through its camera.

[0080] It should be noted that in extreme cases, even after the unmanned device passes through all observation points, it may still fail to obtain image data that meets the set quality requirements. To address this, the unmanned device can increase the angle of view of the target object by performing 360-degree circular motion. If the unmanned device is a drone, the flight altitude can be lowered to reduce the distance to the target object and increase the angle of view. If the unmanned device is an unmanned vehicle, the vehicle's posture can be changed to increase the angle of view of the target object.

[0081] Secondly, the unmanned vehicle can use the image data collected at several observation points of the observed target object while traveling along the observation trajectory to determine an observation point that meets the set conditions from among the observation points of the observed target object, and finally determine the tracking path based on the target point.

[0082] In an embodiment of the present specification, the unmanned device can obtain, for each observation point, the image data collected by the unmanned device for the target object at the observation point, as the image data corresponding to the observation point. Then, the image quality of the image data corresponding to the observation point is determined. The greater the impact of sunlight reflection on the image data, the worse the image quality corresponding to the image data. If the image quality of the image data corresponding to the observation point is greater than a set image quality threshold, the observation point is determined to be an observation point that meets the set conditions, as the target point, and stops driving along the observation trajectory. The set image quality threshold mentioned here can be the image quality corresponding to the image data that is predetermined and can accurately identify specific information on the target object.

[0083] In other words, the set condition can be that the image quality of the image data corresponding to the observation point is greater than a set image quality threshold. Based on this, the unmanned vehicle does not need to complete the entire observation trajectory. During the observation trajectory, if image data that can determine specific information about the target object is collected, the unmanned vehicle can stop driving along the observation trajectory and, based on the specific information about the target object, determine a tracking path for tracking the target object from the location where the image data was collected, and control itself according to the tracking path.

[0084] For example, if the unmanned device is a drone and the target object is a marker with a QR code, the drone can fly from the location where the image data was collected in the direction of the marker and land at the marker's location. For another example, if the unmanned device is an unmanned vehicle and the target object is a road sign, the unmanned vehicle can drive in the direction of the road sign based on the specific information on the road sign (such as "go straight 200 meters," "turn left," or "stop") and plan its next driving trajectory according to the specific information on the road sign. If the unmanned device is a drone and the target object is a vehicle's license plate, the drone can track the vehicle from the location where the image data was collected.

[0085] Furthermore, to ensure the accuracy of image data recognition and accurately locate or track a target object, the unmanned device can obtain image data of the target object at all observation points along its observation trajectory, thereby constructing an image set corresponding to the observation trajectory. The unmanned device can then determine the image data with the highest image quality from the image set as the optimal image data, and determine the observation point where the unmanned device captured the optimal image data as the target point.

[0086] In other words, the set condition can be the highest-quality image data determined from the image set. Based on this, the unmanned vehicle needs to complete the entire observation trajectory and determine the highest-quality image data from the collected image set to ensure image data recognition accuracy, thereby allowing the unmanned vehicle to accurately dock at the target object or track the target object.

[0087] In an embodiment of this specification, if the unmanned device continuously captures image data with image quality exceeding a set image quality threshold, the area where the unmanned device continuously captures image data with image quality exceeding the set image quality threshold is deemed to be less affected by sunlight reflection. The unmanned device can use the location of the last frame of image data continuously captured with image quality exceeding the set image quality threshold as the target point. In other words, the set condition can be the last frame of image data continuously captured with image quality exceeding the set image quality threshold.

[0088] Of course, the unmanned device can also plan its own tracking trajectory for the target object based on several historical observation points obtained in the past. Specifically, the unmanned device can determine several historical observation points used to observe the target object based on the lighting information of the environment in which it collected the image data and the observation position. Then, based on these historical observation points, it plans its own tracking path for tracking the target object. In other words, if the unmanned device determines that the light spot in the image data it collected is large, the unmanned device can directly obtain several historical observation points of the target object to plan its own tracking path for tracking the target object.

[0089] In the embodiments of this specification, the unmanned device can obtain historical image data collected by each unmanned device and then determine a number of historical observation points used by the unmanned device to observe the target object based on the historical image data. The historical observation points mentioned here can refer to trajectory points on the historical observation trajectory obtained by using the distance from the observation position in the historical image data to the target object as the radius and the target object as the center.

[0090] Specifically, the unmanned device can use the A* algorithm (A-Star) to ensure that the unmanned device passes through all historical observation points. For details, please refer to the following formula:

[0091] F(n)=G(n)+h s H(n)

[0092] In the above formula, F(n) can be used to represent the distance from the location of the unmanned device to the target object via observation point n, G(n) can be used to represent the actual distance from the location of the unmanned device to observation point n, and H(n) is the predicted distance value of the shortest path from observation point n to the target object. In order to ensure that the unmanned device passes through all historical observation points, the unmanned device can pre-set h s As the unmanned device approaches the target object, it gradually increases. That is, when the unmanned device reaches a certain distance from the target object, the distance the unmanned device travels in the direction away from the target object will be less than the distance the unmanned device travels in the direction close to the target object, thereby ensuring that the unmanned device will not reach the target object before passing all historical observation points. Until it is determined that the unmanned device has passed all historical observation points, h s becomes 0, and the unmanned device can determine the shortest path to the location of the target object.

[0093] Furthermore, the unmanned equipment can also determine the historical observation point with the shortest distance between the location of the unmanned equipment and the historical observation points of several observed targets based on the location of the unmanned equipment and the historical observation points of the observed targets, as the distance-optimal observation point. Secondly, based on the location of the unmanned equipment and the distance-optimal observation point, the first planned trajectory of the unmanned equipment is determined. The first planned trajectory mentioned here is used to characterize the trajectory of the unmanned equipment from the location of the unmanned equipment to the distance-optimal observation point. Then, based on the distance-optimal observation point, other historical observation points and the location of the target object, the second planned trajectory of the unmanned equipment is determined. The second planned trajectory mentioned here is used to characterize the trajectory of the unmanned equipment at the distance-optimal observation point, based on the historical observation points, to the location of the target object. Finally, based on the first planned trajectory and the second planned trajectory, the planned trajectory to the location of the target object is determined.

[0094] The unmanned vehicle may first travel along the second planned trajectory, and then, based on the image data collected during travel along the second planned trajectory, determine image data with image quality exceeding a set image quality threshold. Finally, based on the location at which the image data was collected and the location of the target object, determine the trajectory from the location at which the unmanned vehicle collected the image data to the location of the target object, thereby determining the planned trajectory of the unmanned vehicle to the location of the target object.

[0095] In practical applications, light spots caused by strong light such as sunlight only appear at specific angles during different time periods each day. Therefore, the shooting angle that has less impact on the image quality of the image data remains almost unchanged. Unmanned equipment can determine several better historical observation points based on the image quality of historical image data to shorten the length of the unmanned equipment's observation trajectory, thereby improving the efficiency of the unmanned equipment in performing driving tasks.

[0096] In an embodiment of the present specification, the unmanned device can obtain historical image data collected by the unmanned device at each historical observation point. If the image quality corresponding to the historical image data exceeds a set historical image quality threshold, it is assumed that the image quality of the historical image data is less affected by the light spots caused by sunlight reflection under relatively strong sunlight conditions. Furthermore, the locations at which the unmanned device collected each piece of historical image data with image quality exceeding the set historical image quality threshold are statistically calculated and used as preferred historical observation points. Based on the determined preferred historical observation points, the unmanned device then plans a tracking path for tracking the target object.

[0097] In practical applications, as time changes, the light intensity and angle within the same time period of the day may also change. Therefore, unmanned equipment can determine several preferred historical observation points that change over time based on the image quality of historical image data and different times (dates, seasons, etc.).

[0098] In the embodiments of this specification, the image quality of historical image data can be used to count historical image data with image quality exceeding a set historical image quality threshold during different time periods on each date of the year, and the location of the unmanned device when the historical image data was collected can be determined. Furthermore, based on the current time of the unmanned device, several preferred historical observation points corresponding to the current time can be determined.

[0099] In practical applications, unmanned vehicles may need to perform tasks in multiple observation areas. To ensure mission success, the unmanned vehicle may not proceed to the next area until it has completed its current mission. Therefore, in the embodiments of this specification, the unmanned vehicle can plan a tracking path based on several historical observation points, passing through all of them. After the unmanned vehicle has determined that it has completed the tracking path, it controls itself to proceed to the next observation area.

[0100] As can be seen from the above method, this method can use an observation trajectory containing several observation points of the observation target object to shoot the target object at different angles through the camera to find the shooting angle that has less impact on the image quality of the image data, and improve the image quality of the collected image data through this method. The image data corresponding to the shooting angle is identified to increase the success rate of image recognition. Then, based on the location at which the image data was collected and the location of the target object, the tracking path of the unmanned device for tracking the target object from the location at which the image data meeting the set conditions was collected or the planned trajectory for accurately docking at the location of the target object is determined, and the unmanned device is controlled. Based on this, the method can plan the tracking path based on the lighting information of the environment in which the unmanned device is located when collecting image data and the observation position, thereby increasing the shooting range of the unmanned device for collecting image data, allowing the unmanned device to collect image data with higher image quality, and then better identify the target object in the image data, thereby better tracking the target object or accurately docking at the location of the target object.

[0101] The above is a control method for unmanned equipment provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding control device for unmanned equipment, such as Figure 3 shown.

[0102] Figure 3This is a schematic diagram of a control device for unmanned equipment provided in this specification. The method is applied to the field of unmanned driving and specifically includes:

[0103] An acquisition module 300 is used to acquire image data containing a target object collected by an unmanned device;

[0104] A determination module 302 is configured to determine an observation position corresponding to the target object based on the image data;

[0105] A tracking module 304 is configured to plan a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position;

[0106] The control module 306 is configured to control the unmanned equipment according to the tracking path.

[0107] Optionally, the tracking module 304 is specifically used to determine the position where the unmanned equipment is located when it collects image data that meets the set conditions based on the lighting information of the environment in which the unmanned equipment is located when collecting the image data and the observation position, as the target point, and plan the tracking path for the unmanned equipment to track the target object based on the target point.

[0108] Optionally, the tracking module 304 is specifically used to determine the image quality of the image data collected by the unmanned equipment based on the lighting information of the environment when the unmanned equipment collects the image data and the observation position, and the position where the image quality is greater than the set image quality threshold is used as the target point.

[0109] Optionally, the tracking module 304 is specifically used to determine an observation trajectory including several observation points for observing the target object based on the lighting information of the environment in which the unmanned equipment is located when collecting the image data and the observation position, and according to the image data collected at the several observation points for observing the target object when the unmanned equipment is traveling along the observation trajectory, determine an observation point that meets the set conditions from the several observation points for observing the target object as the target point, and determine the tracking path based on the target point.

[0110] Optionally, the tracking module 304 is specifically used to obtain, for each observation point, image data collected by the unmanned equipment for the target object at the observation point as the image data corresponding to the observation point, and determine the image quality of the image data corresponding to the observation point; if the image quality of the image data corresponding to the observation point is greater than a set image quality threshold, determine that the observation point is an observation point that meets the set conditions, serve as the target point, and stop driving along the observation trajectory.

[0111] Optionally, the tracking module 304 is specifically used to obtain the image data collected by the unmanned equipment for the target object at all observation points in the observation trajectory according to the observation trajectory, so as to construct an image set corresponding to the observation trajectory, and determine the image data with the highest image quality from the image set as the optimal image data, and determine the observation point where the unmanned equipment collects the optimal image data as the target point.

[0112] Optionally, the tracking module 304 is specifically used to determine several historical observation points used by the unmanned equipment to observe the target object based on the lighting information of the environment in which the unmanned equipment is located when collecting the image data and the observation position, and plan the tracking path for the unmanned equipment to track the target object based on the several historical observation points.

[0113] Optionally, the tracking module 304 is specifically used to obtain historical image data collected by unmanned equipment at each historical observation point in history. If it is determined that the image quality corresponding to the historical image data is greater than a set historical image quality threshold, the historical observation point is used as a preferred historical observation point. Based on the determined preferred historical observation point, the tracking path for the unmanned equipment to track the target object is planned.

[0114] Optionally, the tracking module 304 is specifically used to plan a tracking path passing through all historical observation points based on the multiple historical observation points, and after determining that the unmanned device has completed the tracking path, control the unmanned device to go to the next observation area.

[0115] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 A control method for unmanned equipment is provided.

[0116] This manual also provides Figure 4 The schematic structure diagram of the unmanned equipment is shown in FIG. Figure 4 As mentioned above, at the hardware level, the unmanned equipment includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0117] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0118] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0119] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0120] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0121] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0126] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0127] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0128] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0129] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0131] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0132] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A control method for unmanned equipment, characterized in that: The method is applied to the field of unmanned driving, including: Acquire image data containing target objects collected by unmanned equipment; determining an observation position corresponding to the target object based on the image data; Planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position; Controlling the unmanned equipment according to the tracking path; The step of planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position includes: The tracking path of the collected image data with a smaller light spot is determined according to the light spot size in the collected image data and the observation position.

2. The method according to claim 1, wherein Planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, specifically including: Determining, based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, the position at which the unmanned device is located when collecting the image data that meets the set conditions as the target point; According to the target point, a tracking path for the unmanned equipment to track the target object is planned.

3. The method according to claim 2, wherein Determining, based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, the position at which the unmanned device collects the image data that meets the set conditions as the target point, specifically includes: According to the lighting information of the environment in which the unmanned equipment is located when collecting the image data and the observation position, the position where the image quality of the image data collected by the unmanned equipment is greater than the set image quality threshold is determined as the target point.

4. The method according to claim 1, wherein Planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, specifically including: determining an observation trajectory including a plurality of observation points for observing the target object based on lighting information of an environment in which the unmanned equipment is located when collecting the image data and the observation position; determining, based on image data collected at the plurality of observation points for observing the target object when the unmanned device travels along the observation trajectory, an observation point that meets set conditions from the plurality of observation points for observing the target object as a target point; The tracking path is determined according to the target point.

5. The method according to claim 4, wherein Determining, based on image data collected at the plurality of observation points for observing the target object when the unmanned device travels along the observation trajectory, an observation point that meets set conditions from the plurality of observation points for observing the target object as a target point, specifically includes: For each observation point, image data collected by the unmanned equipment at the observation point for the target object is obtained as image data corresponding to the observation point; Determining the image quality of the image data corresponding to the observation point; If the image quality of the image data corresponding to the observation point is greater than the set image quality threshold, the observation point is determined to be an observation point that meets the set conditions, is used as the target point, and the vehicle stops traveling along the observation trajectory.

6. The method according to claim 4, wherein Determining, based on image data collected at the plurality of observation points for observing the target object when the unmanned device travels along the observation trajectory, an observation point that meets set conditions from the plurality of observation points for observing the target object as a target point, specifically includes: Acquire, according to the observation trajectory, image data collected by the unmanned equipment for the target object at all observation points in the observation trajectory, so as to construct an image set corresponding to the observation trajectory; From the image set, the image data with the highest image quality is determined as the optimal image data, and the observation point where the unmanned equipment collects the optimal image data is determined as the target point.

7. The method according to claim 1, wherein Planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position, specifically including: determining, based on the lighting information of the environment in which the unmanned equipment was located when collecting the image data and the observation position, a number of historical observation points used by the unmanned equipment to observe the target object; A tracking path for the unmanned equipment to track the target object is planned based on the multiple historical observation points.

8. The method according to claim 7, wherein Planning a tracking path for the unmanned device to track the target object based on the plurality of historical observation points specifically includes: For each historical observation point, obtain historical image data collected by unmanned equipment at the historical observation point; If it is determined that the image quality corresponding to the historical image data is greater than the set historical image quality threshold, the historical observation point is used as a preferred historical observation point; Based on the determined preferred historical observation points, a tracking path for the unmanned equipment to track the target object is planned.

9. The method according to claim 7, wherein Planning a tracking path for the unmanned device to track the target object based on the plurality of historical observation points specifically includes: Planning a tracking path passing through all the historical observation points based on the plurality of historical observation points; Controlling the unmanned equipment according to the tracking path specifically includes: After determining that the unmanned equipment has completed the tracking path, the unmanned equipment is controlled to move to the next observation area.

10. A control device for unmanned equipment, characterized in that: The device is applied to the field of unmanned driving, including: An acquisition module is used to acquire image data containing target objects collected by unmanned equipment; a determination module, configured to determine an observation position corresponding to the target object based on the image data; A tracking module, configured to plan a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position; A control module, configured to control the unmanned equipment according to the tracking path; The step of planning a tracking path for the unmanned device to track the target object based on the lighting information of the environment in which the unmanned device is located when collecting the image data and the observation position includes: The tracking path of the collected image data with a smaller light spot is determined according to the light spot size in the collected image data and the observation position.

11. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. An unmanned device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 9 is implemented.

Images