An Unmanned Aerial Vehicle Logistics Control Method, System and Unmanned Aerial Vehicle Based on Image Recognition

Abstract

The present application provides a method, system and drone for controlling drone logistics based on image recognition, which relates to the technical field of warehouse management. The method obtains the current tour field-of-view image captured by the drone at the current shooting point, compares the current tour field-of-view image with the previous tour field-of-view image. If there is a change in the goods, a multi-point recognition instruction is sent to the drone, so that the drone captures auxiliary field-of-view images at each circumferential recognition point corresponding to the current shooting point and uploads them to the cloud for tipping risk recognition. If there is no change in the goods, the next shooting point in the tour path of the current shooting point is sent to the drone. It can reduce the amount of data processed by the cloud, reduce the time for a single inspection by the drone, and thus reduce the cost of warehouse management.

Description

Technical Field

[0001] This application relates to the technical field of warehouse management, and specifically relates to an image recognition-based UAV logistics control method, system, and UAV. Background Art

[0002] Warehouse goods management can use UAVs to inspect the goods in the logistics warehouse, enabling unmanned warehouse management, improving inspection efficiency, and reducing inspection costs.

[0003] In a related art image recognition-based UAV logistics control method, the UAV flies along a preset inspection path. After the UAV flies to any shooting point, it first hovers at the shooting point to capture a panoramic view image, and then flies to each circumferential recognition point corresponding to the shooting point to capture auxiliary view images in sequence. After uploading the panoramic view image and the auxiliary view images to the cloud, the cloud determines whether there is a risk of the goods captured by the UAV being toppled. However, this method requires a large amount of computing power support, which in turn leads to a significant increase in management costs. On the other hand, the UAV itself has limited battery life, and hovering at multiple points may cause an increase in the power consumption of the UAV, increasing the inspection cost of the UAV. Summary of the Invention

[0004] This application provides an image recognition-based UAV logistics control method, system, and UAV, which can reduce the amount of data processed by the cloud, reduce the time for a single UAV inspection, and thus reduce the cost of warehouse management.

[0005] An image recognition-based UAV logistics control method provided by this application includes:

[0006] Obtain the current panoramic view image captured by the UAV at the current shooting point; the UAV is used to fly along a preset tour path and capture the goods stored on the shelves in the logistics warehouse at the current shooting point;

[0007] If there is a previous panoramic view image corresponding to the current shooting point, compare the current panoramic view image with the previous panoramic view image to determine whether there is a change in the goods in the UAV's field of view corresponding to the current shooting point;

[0008] If there is a change in the goods, send a multi-point recognition instruction to the UAV, so that after the UAV receives the multi-point recognition instruction, it captures auxiliary view images at each circumferential recognition point corresponding to the current shooting point and uploads them to the cloud for tipping risk recognition;

[0009] If there is no change in the goods, send the next shooting point in the tour path of the current shooting point to the UAV.

[0010] Optionally, if there are changes in the goods, a multi-point identification instruction is sent to the drone, including:

[0011] If there are changes in the goods, obtain the vibration data uploaded to the cloud by the target vibration sensor corresponding to the current shooting point;

[0012] If the vibration data meets the vibration judgment condition, send a multi-point identification instruction to the drone;

[0013] If the vibration data does not meet the vibration judgment condition, send the next shooting point in the current shooting point of the tour path to the drone.

[0014] Optionally, the vibration data includes acceleration, and the vibration judgment condition includes:

[0015] The peak acceleration within a preset time is greater than the first threshold; and / or,

[0016] The effective value of the acceleration within a preset time is greater than the second threshold.

[0017] Optionally, the target threshold includes the first threshold or the second threshold, and the method further includes:

[0018] Obtain the current load weight of the shelf and the preset load weight of the shelf; the preset load weight of the shelf represents the maximum load weight of the shelf;

[0019] Determine the first adjustment coefficient based on the ratio of the current load weight of the shelf to the preset load weight of the shelf;

[0020] Based on the shelf level where the goods are located in the preset mapping relationship between the shelf level and the adjustment coefficient, determine the second adjustment coefficient;

[0021] Determine the target threshold according to the first adjustment coefficient, the second adjustment coefficient and the preset initial value.

[0022] Optionally, if there are changes in the goods, obtain the vibration data uploaded to the cloud by the target vibration sensor corresponding to the current shooting point, including:

[0023] If there are changes in the goods, send a high-frequency operation instruction to the target vibration sensor, so that the target vibration sensor changes from low-frequency operation to high-frequency operation and operates for a preset time in the high-frequency operation state, and the frequency of the high-frequency operation is 1Hz~10Hz;

[0024] Obtain the vibration data of the target vibration sensor operating for a preset time in the high-frequency operation state.

[0025] Optionally, obtain the current tour field of view image captured by the drone at the current shooting point, including:

[0026] In response to receiving the in-place information sent by the drone, the cloud sends an activation command to the light source device, so that after receiving the activation command, the light source device performs light compensation on the drone's field of view corresponding to the current shooting position, and the in-place information indicates that the drone has flown to the current shooting position.

[0027] Optionally, in response to receiving the in-place information sent by the drone, the cloud sending an activation command to the light source device includes:

[0028] In response to receiving the in-place information sent by the drone, the cloud obtains the natural light intensity;

[0029] If the natural light intensity is less than the light threshold, the cloud sends an activation command to the light source device.

[0030] Another method for controlling an unmanned aerial vehicle (UAV) logistics based on image recognition provided by this application includes:

[0031] In response to receiving the current shooting position sent by the cloud, the drone flies along the patrol path to the current shooting position and captures an image of the current patrol field of view; and uploads the current patrol field of view image to the cloud;

[0032] In response to receiving the multi-position recognition command sent by the cloud, the drone flies to each circumferential recognition position corresponding to the current shooting position to capture auxiliary field of view images, and uploads the auxiliary field of view images to the cloud;

[0033] In response to receiving the next shooting position sent by the cloud, the drone flies along the patrol path to the next shooting position to capture the next patrol field of view image; and uploads the next patrol field of view image to the cloud.

[0034] To achieve the above object and other related objects, this application also provides an unmanned aerial vehicle (UAV) logistics control system based on image recognition, including a cloud, a UAV, and one or more processors. The UAV is communicatively connected to the cloud, and the processor is disposed on the UAV and / or the cloud. The processor is configured to execute one or more of the above-described methods for controlling UAV logistics based on image recognition.

[0035] To achieve the above object and other related objects, this application also provides a UAV, including a processor, and the processor is configured to execute one or more of the above-described methods for controlling UAV logistics based on image recognition.

[0036] As described above, a method, a system, and a UAV for controlling UAV logistics based on image recognition provided by this application have the following beneficial effects:

[0037] A method for controlling UAV logistics based on image recognition in this application. This method obtains the current tour vision image captured by the UAV at the current shooting point, compares the current tour vision image with the previous tour vision image. If there is a change in the goods, a multi-point recognition instruction is sent to the UAV, so that the UAV captures auxiliary vision images at each circumferential recognition point corresponding to the current shooting point and uploads them to the cloud for tipping risk recognition. If there is no change in the goods, the next shooting point in the tour path of the current shooting point is sent to the UAV. This can reduce the amount of data processed by the cloud, reduce the time for a single UAV inspection, and thus reduce the cost of warehouse management.

[0038] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. Brief Description of the Drawings

[0039] The drawings here are incorporated into the specification and form a part of this specification, showing the embodiments in line with this application, and are used together with the specification to explain the principles of this application. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0040] Figure 1 is a schematic diagram of the implementation scenario of the method for controlling UAV logistics based on image recognition shown in an exemplary embodiment of this application;

[0041] Figure 2 is a flowchart of the method for controlling UAV logistics based on image recognition shown in an exemplary embodiment of this application;

[0042] Figure 3 is a schematic diagram of the implementation scenario of the method for controlling UAV logistics based on image recognition shown in another exemplary embodiment of this application;

[0043] Figure 4 is a flowchart of the method for controlling UAV logistics based on image recognition shown in another embodiment of this application. Detailed Description of the Embodiments

[0044] The following will describe the embodiments of this application with reference to the drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for explaining this application, rather than for limiting the protection scope of this application.

[0045] It should be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of the present application. Therefore, only the components related to the present application are shown in the drawings, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0046] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0047] Please refer to Figure 1 , which is a schematic diagram of an implementation scenario of an image recognition-based UAV logistics control method shown in an exemplary embodiment of the present application.

[0048] This implementation scenario may include a UAV 110, a vibration sensor 120, a light source device 130, and a cloud (not shown in the figure). The UAV 110, the vibration sensor 120, and the light source device 130 can all be connected to the cloud wirelessly. Figure 1 The dotted line with an arrow in the figure represents the patrol path of the UAV, and the preset patrol path can be sent from the cloud to the UAV. Multiple shooting points can be set on the patrol path.

[0049] Multiple shelves can be set in the logistics warehouse. After the goods are stored, they can be placed on the shelves. For example, the first goods 200 can be placed in the uppermost right position of the shelf 300.

[0050] The UAV 110 can fly in the logistics warehouse according to the preset patrol path and take pictures at each shooting point. The shooting points can be sent from the cloud to the UAV.

[0051] The vibration sensor 120 can be installed at the bottom of the shelf and upload the obtained vibration data to the cloud.

[0052] The light source device 130 can be installed at the edge of the shelf. The light source device 130 can receive the opening instruction sent from the cloud to perform light compensation on the UAV's field of view, avoiding insufficient light resulting in the cloud being unable to obtain sufficient information from the field of view image.

[0053] It should be noted that each shelf in the logistics warehouse can be installed with a vibration sensor 120 and a light source device 130 ( Figure 1 not shown).

[0054] Please refer toFigure 2 , Figure 2 is a flowchart of an image recognition-based UAV logistics control method shown in an exemplary embodiment of the present application. The image recognition-based UAV logistics control method can be applied to the Figure 1 shown implementation scenario. Referring to Figure 2 , it can be seen that the image recognition-based UAV logistics control method can include:

[0055] Step S210, obtaining a current tour field of view image captured by the UAV at the current shooting point.

[0056] Wherein, the UAV is used to fly according to a preset tour path and capture the goods stored on the shelves in the logistics warehouse at the current shooting point.

[0057] In an embodiment of the present application, the UAV can perform flight inspections in the logistics warehouse according to a preset tour path, hover at the current shooting point, and then capture the current tour field of view image within the UAV's field of view. The UAV's field of view can be the field of view of the UAV camera. When the UAV hovers at the current shooting point, the UAV's field of view can include part of the shelves and at least one good placed on the shelves. The captured current tour field of view image also includes part of the shelves and at least one good placed on the shelves. The current shooting point can be located on the side of the shelf, that is, in front of the goods.

[0058] The cloud can send the current shooting point to the UAV and receive the current tour field of view image captured by the UAV at the current shooting point. The current shooting point and the current tour field of view image correspond.

[0059] It should be noted that the tour field of view images captured by the UAV at all shooting points should include all the goods on the shelves in the logistics warehouse, that is, the UAV captures all the goods in the logistics warehouse. When the UAV hovers at the t-th shooting point, the t-th shooting point is the current shooting point. When the UAV hovers at the (t + 1)-th shooting point, the t-th shooting point is the previous shooting point, and the (t + 1)-th shooting point is the current shooting point. Both the current shooting point and the previous shooting point are shooting points.

[0060] In a possible implementation manner, the process of step S210, obtaining the current tour field of view image captured by the UAV at the current shooting point, can include: in response to receiving the in-place information sent by the UAV, the cloud sends an opening instruction to the light source device, so that the light source device performs light compensation on the UAV's field of view corresponding to the current shooting point after receiving the opening instruction, and the in-place information indicates that the UAV has flown to the current shooting point.

[0061] Among them, in response to receiving the in-place information sent by the drone, the cloud sends an activation command to the light source device, including: in response to receiving the in-place information sent by the drone, the cloud obtains the natural light intensity; if the natural light intensity is less than the light threshold, the cloud sends an activation command to the light source device.

[0062] In a possible implementation, the natural light intensity can be obtained by setting devices such as photosensitive sensors, and the devices such as photosensitive sensors can be wirelessly connected to the cloud. The light threshold can be a preset value designed by the operator.

[0063] In an embodiment of the present application, the light source device can be installed at the edge of the shelf, the light source can be installed at the end of each layer of the shelf, the light source can be installed on the left and right sides of the shelf, and the installation positions of the light sources on a single shelf need to satisfy that the light compensation covers all goods.

[0064] Optionally, if the natural light intensity is less than the light threshold, the cloud can send an activation command to the light source device. The cloud can send an activation command to all light source devices to turn on all light source devices.

[0065] Optionally, the cloud can store the installation shelves of each light source and the positions of each light source. After receiving the in-place information sent by the drone, the cloud can determine the light source device to be activated among multiple light sources according to the current shooting point in the in-place information, and activate the light source device to be activated.

[0066] Exemplarily, please refer to Figure 3 , which is a schematic diagram of an implementation scenario of a drone logistics control method based on image recognition shown in another exemplary embodiment of the present application. In Figure 3 the shown implementation scenario, the drone 110 is located between the first shelf 310 and the second shelf 320, and the drone 110 is located in the space at the top layer of the shelf. At this time, the light source devices installed at the top layer of the first shelf 310 and the second shelf 320 can be turned on, while the light source devices on the third shelf 330 are in the off state.

[0067] Step S220, if there is a previous tour field of view image corresponding to the current shooting point, perform image comparison on the current tour field of view image and the previous tour field of view image to determine whether there are changes in the goods in the field of view of the drone corresponding to the current shooting point.

[0068] In one embodiment of the present application, if there is a previous tour view image corresponding to the current shooting point, the current tour view image and the previous tour view image are compared to determine whether there are any changes in the goods within the drone's field of view corresponding to the current shooting point. The changes in the goods may include at least one of an increase in goods, a decrease in goods, or a change in the position of the goods. The previous tour view image may be stored in a historical database in the cloud. After the tour view images captured by the drone at each shooting point and at each moment are uploaded to the cloud, they can all be stored in the historical database in the cloud.

[0069] It should be noted that the previous tour view image is the tour view image captured by the drone at the previous moment of the current shooting point.

[0070] In a possible implementation, the Mask R-CNN or YOLOv8-Seg model can be used to perform instance segmentation on the previous tour view image and the current tour view image respectively, to obtain the position, category, and pixel-level mask of each good; calculate the IoU (Intersection over Union) of the bounding boxes of the goods in the two images. If IoU > the threshold, it is regarded as the same object; extract the depth features of the goods ROI (Region of Interest) through ResNet-50, calculate the cosine similarity, and associate the same goods in the two images; the existence of unmatched instances in the current tour view image indicates the presence of new goods; the existence of unmatched instances in the previous tour view image indicates the removal of goods; if the instance matching is successful but the position offset is greater than the offset threshold, it indicates a change in the position of the goods.

[0071] Step S230, if there are changes in the goods, send a multi-point recognition instruction to the drone, so that after receiving the multi-point recognition instruction, the drone takes auxiliary view images at each circumferential recognition point corresponding to the current shooting point and uploads them to the cloud for tipping risk recognition.

[0072] In one embodiment of the present application, if there are changes in the goods, the cloud can send a multi-point recognition instruction to the drone. The multi-point recognition instruction may include at least one circumferential recognition point corresponding to the current shooting point, so that after the drone flies to each circumferential recognition point, it takes auxiliary view images at each circumferential recognition point. The cloud can determine whether there is a tipping risk for the goods within the drone's field of view at the current shooting point based on the auxiliary view images.

[0073] It should be noted that the circumferential recognition points may be located above, behind, to the left, or to the right of the goods.

[0074] In one embodiment, after sending a multi-point identification instruction to the drone when there are changes in the goods in step S230, the drone logistics control method based on image recognition may further include: obtaining the auxiliary field-of-view images uploaded by the drone; performing auxiliary dumping identification on the goods within the field of view based on the current tour field-of-view image and each auxiliary field-of-view image to determine whether there is a risk of the goods within the drone's field of view tipping over; the auxiliary field-of-view images are the images captured by the drone at each circumferential identification point corresponding to the current shooting point.

[0075] In an embodiment of the present application, the cloud can perform three-dimensional construction based on the current tour field-of-view image and each auxiliary field-of-view image, analyze the center of gravity of the goods in the drone's field of view, and determine whether there is a risk of the goods tipping over.

[0076] Step S240, if there are no changes in the goods, send the next shooting point in the tour path of the drone to the drone.

[0077] In an embodiment of the present application, if there are no changes in the goods, the cloud can send the next shooting point in the tour path of the drone to the drone, so that after the drone flies to the next shooting point, it takes an image at the next shooting point.

[0078] In the drone logistics control strategy of the related technology, when the drone performs inspection in the logistics warehouse, generally after taking an image at the current shooting point, it takes images at each circumferential identification point corresponding to the current shooting point, and also takes multi-point images when the goods have not changed. On the one hand, the drone needs to hover for a period of time when taking an image at each point, resulting in an increase in the working time of the drone and more power consumption; on the other hand, the cloud needs to perform image recognition on the images at each point to determine whether there is a risk of the goods on the shelf within the drone's field of view tipping over, which requires a high computing power for the cloud, resulting in a high cost.

[0079] In the drone logistics control method based on image recognition provided by the embodiment of the present application, the cloud can determine whether there are changes in the goods within the drone's field of view according to the current tour field-of-view image uploaded by the drone and the previous tour field-of-view image, perform multi-point identification when there are changes in the goods, and when there are no changes in the goods, the drone flies to the next shooting point for inspection. In this way, the amount of data processed by the cloud can be reduced, thereby reducing the processing cost of the cloud. In addition, the number of images captured by the drone is reduced, that is, the hovering time of the drone is reduced, which can reduce the working time of the drone and enable the drone to complete the inspection of the goods with lower power consumption.

[0080] In one embodiment, the process of sending a multi-point identification instruction to the drone when there are changes in the goods in step S230 may include:

[0081] Step S231: If there is a change in the goods, obtain the vibration data uploaded by the target vibration sensor corresponding to the current shooting point to the cloud.

[0082] In an embodiment of the present application, if there is a change in the goods, the vibration data uploaded by the target vibration sensor corresponding to the current shooting point to the cloud can be obtained. The vibration data of each vibration sensor installed on each shelf can be uploaded to the cloud in real time. The vibration data uploaded by each vibration sensor to the cloud can be associated with the shelf number of the shelf where each vibration sensor is located. The current shooting point can correspond to a shelf number. According to the shelf number corresponding to the current shooting point, the cloud can determine the target vibration sensor among multiple vibration sensors based on the shelf number corresponding to the current shooting point and obtain the vibration data corresponding to the target vibration sensor.

[0083] In one embodiment, the process of obtaining the vibration data uploaded by the target vibration sensor corresponding to the current shooting point to the cloud in step S231 when there is a change in the goods may include:

[0084] Step S231A: If there is a change in the goods, send a high-frequency operation instruction to the target vibration sensor to cause the target vibration sensor to change from low-frequency operation to high-frequency operation and operate for a preset time in the high-frequency operation state.

[0085] Among them, the frequency of high-frequency operation is 1Hz to 10Hz.

[0086] In an embodiment of the present application, if there is a change in the goods, the cloud can send a high-frequency operation instruction to the target vibration sensor. In the normal state (that is, when the high-frequency operation instruction is not received), each vibration sensor can operate at a low frequency, and the low-frequency operation range can be (10Hz, 50Hz], that is, the low-frequency operation range can be a range greater than 10Hz and less than or equal to 50Hz. After receiving the high-frequency operation instruction, the target vibration sensor can change from low-frequency operation to high-frequency operation.

[0087] When the vibration sensor operates at a low frequency, the sampling rate is low, the energy consumption is less, and the data storage and transmission burden can be reduced. In the high-frequency operation state of the target vibration sensor, the target vibration sensor can obtain more data so that the cloud can accurately identify whether the shelf vibrates.

[0088] Step S231B: Obtain the vibration data of the target vibration sensor operating for a preset time in the high-frequency operation state.

[0089] In an embodiment of the present application, the cloud can obtain the vibration data of the target vibration sensor operating for a preset time in the high-frequency operation state. The target vibration sensor operates for a preset time in the high-frequency operation state, and during this period, the target vibration sensor can upload the vibration data to the cloud in real time.

[0090] It should be noted that after the target vibration sensor uploads the vibration data during the high-frequency operation state for a preset time to the cloud, the target vibration sensor can be switched from high-frequency operation to low-frequency operation to avoid continuous high energy consumption.

[0091] Step S232, if the vibration data meets the vibration judgment condition, send a multi-point identification instruction to the drone.

[0092] In an embodiment of the present application, if the vibration data meets the vibration judgment condition, the cloud can send a multi-point identification instruction to the drone. If the vibration data meets the vibration judgment condition, it can be determined that the shelf photographed by the drone at the current photographing position vibrates. When the shelf vibrates, the cloud can send a multi-point identification instruction to the drone so that the drone performs three-dimensional reconstruction based on the auxiliary vision image uploaded by the drone to determine whether the goods on the shelf vibrate.

[0093] In a possible implementation manner, the vibration data includes acceleration, and the vibration judgment condition includes:

[0094] The peak acceleration within a preset time is greater than a first threshold; and / or, the effective value of the acceleration within a preset time is greater than a second threshold.

[0095] It should be noted that the first threshold and the second threshold can be preset values set by designers according to historical experience. When the peak acceleration within the preset time is greater than the first threshold, it indicates that the shelf vibrates. When the effective value of the acceleration within the preset time is greater than the second threshold, it indicates that the shelf vibrates.

[0096] Exemplarily, the preset time is 30s.

[0097] In an embodiment, the target threshold includes the first threshold or the second threshold, and the method further includes: obtaining the current load weight of the shelf and the preset load weight of the shelf; the preset load weight of the shelf represents the maximum load weight of the shelf; determining a first adjustment coefficient based on the ratio of the current load weight of the shelf to the preset load weight of the shelf; determining a second adjustment coefficient based on the shelf level where the goods are located in the preset mapping relationship between the shelf levels and the adjustment coefficients; determining the target threshold according to the first adjustment coefficient, the second adjustment coefficient and the preset initial value.

[0098] In one embodiment of the present application, the cloud can store the current load weight of the shelf and the preset load weight of the shelf. When goods are warehoused, the current load weight of the shelf can be stored in the cloud in advance. The current load weight of the shelf represents the total weight of the goods currently stored on the shelf. The ratio of the current load weight of the shelf to the preset load weight of the shelf can be determined as the first adjustment coefficient. Generally, the current load weight of the shelf is less than or equal to the preset load weight of the shelf. Based on the shelf level where the goods are located, the second adjustment coefficient is determined in the preset mapping relationship between the shelf levels and the adjustment coefficients. When the drone acquires the current panoramic view image, it can correspondingly send the drone height to the cloud, and based on the drone height and the preset height of each shelf level, determine the shelf level where the goods are located, and determine the second adjustment coefficient in the preset mapping relationship between the shelf levels and the adjustment coefficients. The product of the first adjustment coefficient, the second adjustment coefficient, and the preset initial value can be determined as the target threshold.

[0099] It should be noted that as the shelf level increases, the second adjustment coefficient gradually decreases. For example, when the shelf level is the first layer (i.e., the bottom layer), the second adjustment coefficient can be 1, and when the shelf level is the second layer, the second adjustment coefficient can be 0.8. Such a setting can associate the target threshold with the shelf level. The higher the shelf level, the more likely it is to vibrate, and the smaller the target threshold.

[0100] Exemplarily, when the target threshold is the first threshold, the preset initial value can be 0.5, and when the target threshold is the second threshold, the preset initial value can be 0.2.

[0101] Step S233, if the vibration data does not meet the vibration judgment condition, send the next shooting point in the tour path of the current shooting point to the drone.

[0102] In one embodiment of the present application, if the vibration data does not meet the vibration judgment condition, it can indicate that the shelf corresponding to the current shooting point of the drone has not vibrated. When the shelf does not vibrate and the goods have not changed, generally, the goods do not have the risk of tipping over. At this time, the cloud can send the next shooting point in the tour path of the current shooting point to the drone.

[0103] Please refer to Figure 4 , which is a flowchart of the drone logistics control method based on image recognition shown in another embodiment of the present application. Refer to Figure 4 It can be seen that the drone logistics control method based on image recognition can include:

[0104] Step S410, in response to receiving the current shooting point sent by the cloud, the drone flies along the tour path to the current shooting point and takes the current panoramic view image; and uploads the current panoramic view image to the cloud.

[0105] Step S420: In response to receiving the multi-point recognition instruction sent by the cloud, the drone flies to each circumferential recognition point corresponding to the current shooting point to capture auxiliary vision images, and uploads the auxiliary vision images to the cloud.

[0106] Step S430: In response to receiving the next shooting point sent by the cloud, the drone flies along the tour path to the next shooting point to capture the next tour vision image; and uploads the next tour vision image to the cloud.

[0107] An embodiment of the present application further provides a drone logistics control system, including: a cloud, a drone, and one or more processors. The drone is communicatively connected to the cloud, and the processors are disposed in the drone and / or the cloud. The processors are configured to execute the drone logistics control method based on image recognition provided in each of the above embodiments.

[0108] On the other hand, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer is caused to execute the drone logistics control method based on image recognition provided in each of the above embodiments. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist separately without being assembled into the electronic device.

[0109] On the other hand, the present application further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the drone logistics control method based on image recognition provided in each of the above embodiments.

[0110] On the other hand, the present application further provides a drone, which includes a processor configured to execute the drone logistics control method based on image recognition provided in each of the above embodiments.

[0111] In the embodiments of the present application, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. The terms "comprising" and "including" mentioned throughout the specification and claims are open-ended terms and should be construed as "including but not limited to".

[0112] The above embodiments are only illustrative of the principles and effects of the present application and are not intended to limit the present application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in the present application should still be covered by the claims of the present application.

Claims

1. A method for controlling an unmanned aerial vehicle (UAV) logistics based on image recognition, characterized in that, Including: Obtain the current tour field of view image captured by the drone at the current shooting position; The drone is used to fly along a preset tour path and capture the goods stored on the shelves in the logistics warehouse at the current shooting position; If there is a previous tour field of view image corresponding to the current shooting position, compare the current tour field of view image with the previous tour field of view image to determine whether there is a change in the goods in the field of view of the drone corresponding to the current shooting position; If there is a change in the goods, send a multi-point identification instruction to the drone, so that after receiving the multi-point identification instruction, the drone captures auxiliary field of view images at each circumferential identification position corresponding to the current shooting position and uploads them to the cloud for tipping risk identification; If there is no change in the goods, send the next shooting position in the tour path corresponding to the current shooting position to the drone; If there is a change in the goods, send a multi-point identification instruction to the drone, including: If there is a change in the goods, obtain the vibration data uploaded to the cloud by the target vibration sensor corresponding to the current shooting position; If the vibration data meets the vibration judgment condition, send a multi-point identification instruction to the drone; If the vibration data does not meet the vibration judgment condition, send the next shooting position in the tour path corresponding to the current shooting position to the drone.

2. The method for controlling an unmanned aerial vehicle logistics based on image recognition according to claim 1, wherein, The vibration data includes acceleration, and the vibration judgment condition includes: The peak acceleration within a preset time is greater than the first threshold; and / or, The effective value of the acceleration within a preset time is greater than the second threshold.

3. The method for controlling an unmanned aerial vehicle logistics based on image recognition according to claim 2, wherein The target threshold includes the first threshold or the second threshold, and the method further includes: Obtain the current load weight of the shelf and the preset load weight of the shelf; the preset load weight of the shelf represents the maximum load weight of the shelf; Determine the first adjustment coefficient based on the ratio of the current load weight of the shelf to the preset load weight of the shelf; Determine the second adjustment coefficient based on the shelf level where the goods are located in the preset mapping relationship between the shelf level and the adjustment coefficient; Determine the target threshold according to the first adjustment coefficient, the second adjustment coefficient and the preset initial value.

4. The method for controlling an unmanned aerial vehicle logistics based on image recognition according to claim 1, wherein, If there is a change in the goods, obtain the vibration data uploaded to the cloud by the target vibration sensor corresponding to the current shooting position, including: If there is a change in the goods, send a high-frequency operation instruction to the target vibration sensor, so that the target vibration sensor changes from low-frequency operation to high-frequency operation and operates for a preset time in the high-frequency operation state, and the frequency of the high-frequency operation is 1Hz~10Hz; Obtain the vibration data of the target vibration sensor operating for a preset time in the high-frequency operation state.

5. The method for controlling an unmanned aerial vehicle (UAV) logistics based on image recognition according to any one of claims 1-4, wherein Obtain the current tour field of view image captured by the drone at the current shooting position, including: In response to receiving the in-place information sent by the drone, the cloud sends an opening instruction to the light source device, so that after receiving the opening instruction, the light source device compensates the light for the field of view of the drone corresponding to the current shooting position, and the in-place information indicates that the drone has flown to the current shooting position.

6. The method for controlling an unmanned aerial vehicle (UAV) logistics based on image recognition according to claim 5, wherein In response to receiving the in-place information sent by the drone, the cloud sends an opening instruction to the light source device, including: In response to receiving the in-place information sent by the drone, the cloud obtains the natural light intensity; If the natural light intensity is less than the light threshold, the cloud sends an opening instruction to the light source device.

7. A method for controlling an unmanned aerial vehicle (UAV) logistics based on image recognition, characterized in that, Including: In response to receiving the current shooting point sent by the cloud, the drone flies along the tour path to the current shooting point and then shoots the current tour field of view image; and uploads the current tour field of view image to the cloud; In response to receiving the multi-point recognition instruction sent by the cloud, the drone flies to each circumferential recognition point corresponding to the current shooting point to shoot the auxiliary field of view image, and uploads the auxiliary field of view image to the cloud; In response to receiving the next shooting point sent by the cloud, the drone flies along the tour path to the next shooting point to shoot the next tour field of view image; and uploads the next tour field of view image to the cloud; The method further includes: If there is a previous tour field of view image corresponding to the current shooting point, compare the current tour field of view image with the previous tour field of view image to determine whether there is any change in the goods in the field of view of the drone corresponding to the current shooting point; If there is a change in the goods, obtain the vibration data uploaded by the target vibration sensor corresponding to the current shooting point to the cloud; if the vibration data meets the vibration judgment condition, send a multi-point recognition instruction to the drone; if the vibration data does not meet the vibration judgment condition, send the next shooting point of the current shooting point in the tour path to the drone.

8. An unmanned aerial vehicle logistics control system based on image recognition, characterized in that, It includes a cloud, a drone and one or more processors. The drone is communicatively connected to the cloud. The processor is provided in the drone and / or the cloud. The processor is used to execute the image recognition-based drone logistics control method according to any one of claims 1-7.

9. A drone, characterized in that, It includes a processor. The processor is used to execute the image recognition-based drone logistics control method according to any one of claims 1-7.

Images