Article detection method and device, electronic equipment and storage medium

Abstract

The invention discloses an article detection method and device, electronic equipment and a storage medium. The method is applied to a distribution robot, the distribution robot comprises detachable storage layers, and each storage layer is associated with a camera. The method comprises the following steps: acquiring storage layer change information of the distribution robot; according to the storage layer change information, the associated cameras of the current remaining storage layers of the distribution robot and the detection distances of the associated cameras are determined again; according to the detection distance of the associated camera, adjusting a storage layer image shot by the associated camera; and based on the adjusted image of the object placing layer, detecting the object in the object placing layer. According to the scheme, the detachable storage layer is arranged on the robot, and by disassembling the storage layer, the problem that the robot cannot distribute objects with the height larger than that of the storage layer is solved; after the object placing layer is adjusted, the newest detection distance of the camera is determined, and the shot image is adjusted according to the detection distance so as to determine object characteristics, so that the accuracy of object detection based on the adjusted image is ensured.

Description

Technical Field

[0001] The present invention relates to the field of robots, and in particular to an object detection method, device, electronic equipment and storage medium. Background Art

[0002] With the development of intelligent technology, today's service industry, especially service industries in complex scenarios such as hotels and restaurants, increasingly use robots to deliver items. Delivery robots can deliver food and other items to customers according to set routes.

[0003] Currently, existing delivery robots do not support the disassembly of storage layers, which makes it impossible for the delivery robots to deliver items that are greater than the height of the storage layers, resulting in limited items that the delivery robots can deliver. Summary of the invention

[0004] The present invention provides an object detection method, device, electronic equipment and storage medium.

[0005] According to one aspect of the present invention, there is provided an object detection method, which is applied to a delivery robot, wherein the delivery robot comprises a plurality of storage layers including at least one detachable storage layer, and each storage layer is respectively associated with a camera for collecting images of the storage layer; the method comprises:

[0006] Obtaining storage layer change information of the delivery robot;

[0007] Re-determining the associated cameras of the respective remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information;

[0008] According to the detection distance of the associated camera, adjust the storage layer image taken by the associated camera;

[0009] Based on the adjusted storage layer image, the objects in the storage layer are detected.

[0010] According to another aspect of the present invention, there is provided an object detection device, which is configured on a delivery robot, wherein the delivery robot includes a plurality of storage layers including at least one detachable storage layer, and each storage layer is associated with a camera for collecting images of the storage layer; the device includes:

[0011] A storage layer change information acquisition module, used to acquire storage layer change information of the delivery robot;

[0012] An association relationship determination module, used to re-determine the associated cameras of the respective remaining storage layers of the delivery robot and the detection distance of the associated cameras according to the storage layer change information;

[0013] An image adjustment module, used for adjusting the storage layer image taken by the associated camera according to the detection distance of the associated camera;

[0014] The detection module is used to detect the objects in the storage layer based on the adjusted storage layer image.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the object detection method described in the embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the object detection method described in the embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention is to set a detachable storage layer on the robot, and solve the problem that the robot cannot deliver items that are larger than the height of the storage layer by adjusting the storage layer; and after adjusting the storage layer, the latest detection distance of the camera is determined in time, and the captured image is adjusted according to the detection distance to clarify the object features, thereby ensuring the accuracy of object detection based on the adjusted image.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1a is a schematic diagram of the hardware structure of a delivery robot provided according to an embodiment of the present invention;

[0024] Figure 1b is a schematic diagram of an interactive interface of a delivery robot provided according to an embodiment of the present invention;

[0025] Figure 1c is a flow chart of an object detection method provided according to Embodiment 1 of the present invention;

[0026] Figure 2 is a flow chart of an object detection method provided according to Embodiment 2 of the present invention;

[0027] Figure 3 is a flow chart of an object detection method provided according to Embodiment 3 of the present invention;

[0028] Figure 4 is a flow chart of an object detection method provided according to Embodiment 4 of the present invention;

[0029] Figure 5 is a flow chart of an object detection method provided according to Embodiment 5 of the present invention;

[0030] Figure 6 is a structural schematic diagram of an object detection device provided according to Embodiment 6 of the present invention;

[0031] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the object detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] First, the hardware structure of the delivery robot in the present invention is described. The delivery robot of the present invention includes at least a mobile chassis for realizing the movement / turning / stopping of the robot. Above the mobile chassis, multiple storage layers including at least one detachable storage layer are arranged in the vertical direction. Among them, each storage layer includes a tray for carrying items to be delivered. The tray is only an example, and it can also be other components for carrying items, which are not specifically limited here. Each storage layer is respectively associated with a camera (such as an RGB camera) for capturing images of the storage layer. Optionally, a camera for capturing images of the storage layer is deployed directly above or above the side of the tray of each storage layer. In the specific deployment, for any storage layer, the associated camera of the storage layer can be set at the bottom of the target storage layer adjacent to the storage layer and above the storage layer.

[0034] In an optional implementation scheme, taking the delivery robot as a food delivery robot in a restaurant scenario as an example, its hardware structure diagram can be found in the attached Figure 1a In addition to the mobile chassis, the delivery robot also includes four storage layers, which are marked with numbers 1, 2, 3, and 4 respectively; a camera a for collecting images in storage layer 1 is arranged on the upper side of the tray of storage layer 1, that is, storage layer 1 is associated with camera a, and camera a can be arranged below the image module on the top of the robot, that is, camera a is non-detachable, which can ensure that the robot always has at least one layer for item recognition, thereby improving user experience; a camera b for collecting images in storage layer 2 is arranged directly above the tray of storage layer 2, that is, storage layer 2 is associated with camera b; a camera c for collecting images in storage layer 3 is arranged directly above the tray of storage layer 3, that is, storage layer 3 is associated with camera c. Among them, the storage layer 1 on the top layer is a non-detachable storage layer, and stereoscopic vision sensors m are arranged on both sides of the non-detachable storage layer 1, which are used to collect point cloud data of the surrounding environment of the delivery robot, so that the delivery robot can move laterally and avoid obstacles according to the point cloud data of the surrounding environment; storage layer 2 and storage layer 3 are detachable storage layers, so that by disassembling storage layer 2 and / or storage layer 3, it is ensured that the delivery robot can deliver items with a height greater than the height of the storage layer.

[0035] In the scheme of the present invention, the storage layer disassembly process can be as follows: first, before disassembling the storage layer, the delivery robot is controlled to shut down; then the user selects the storage layer to be disassembled according to the properties of the items to be delivered (such as height), and disassembles it; wherein disassembling the storage layer actually removes the tray and related components of the storage layer to leave more space; after the disassembly is completed, the delivery robot is controlled to start up, and after the delivery robot is turned on, the user can input information such as the number of storage layers to be disassembled and the number of remaining storage layers through the interactive interface provided by the display device (such as a touch control screen) of the delivery robot. It is also possible to disassemble the storage layer when the robot is turned on, and on the robot's setting page, the user can input information such as the number of storage layers to be disassembled and the number of remaining storage layers through the interactive interface provided by the display device (such as a touch control screen) of the delivery robot.

[0036] For example, see the attached Figure 1b, which shows a schematic diagram of an interactive interface, on which the number of storage layers is displayed, and two functional areas are provided. The first functional area provides a storage layer automatic identification function; if the user triggers the storage layer automatic identification function, the delivery robot automatically detects which storage layer is disassembled and updates the number of storage layers; when the automatic identification function is turned on, if the robot detects a storage layer change, it determines whether it is in a task state, such as whether it is performing a meal delivery task. If it is in a task state, the storage layer will not be updated at this time to avoid confusion in task execution. When the robot completes the task, it updates the storage layer according to the latest detected storage layer number.

[0037] The second functional area provides a function for manually selecting to remove a storage layer. Optionally, a simplified diagram of the delivery robot is displayed in the second functional area, and each detachable storage layer of the delivery robot corresponds to a control for the user to select. If the user does not trigger the automatic storage layer identification function, the user can directly select the storage layer to be removed by touching the corresponding control of the detachable storage layer, and then the delivery robot responds to the user's touch operation and updates the number of storage layers. For example Figure 1b Originally there were 4 storage layers. After removing one storage layer, the number of storage layers became 3.

[0038] Based on the above-mentioned delivery robot hardware, the specific process of the object detection method of the present invention can be referred to the introduction of the following embodiment.

[0039] Embodiment 1

[0040] Figure 1c A flowchart of an object detection method is provided for the first embodiment of the present invention. This embodiment is applicable to scenarios where robots are used to deliver objects. Typically, it is applicable to scenarios where robots are used to deliver food in restaurants. The method can be performed by an object detection device. The object detection device can be implemented in the form of hardware and / or software. The object detection device can be configured in an electronic device, such as integrated in a robot device.

[0041] like Figure 1c As shown, the object detection method includes:

[0042] S101. Obtaining storage layer change information of the delivery robot.

[0043] In an embodiment of the present invention, the storage layer change information of the delivery robot may optionally be the relevant information of the storage layer of the delivery robot that has been disassembled, such as the identification information of the disassembled storage layer and the information of the associated camera corresponding to the disassembled storage layer. In an optional implementation, if the user triggers the automatic storage layer recognition function in the interactive interface, the storage layer change information of the delivery robot is determined according to the automatic storage layer recognition result; if the user does not trigger the automatic storage layer recognition function in the interactive interface, but directly selects the disassembled storage layer, the relevant information of the disassembled storage layer is determined according to the identified user triggering operation of the corresponding control of the detachable storage layer. Exemplarily, the automatic recognition function can be implemented by setting a detection sensor at the connection between the robot body and the tray. When the tray is not disassembled, the circuit is connected, and it can be determined that the storage layer is installed with a tray. When the tray is disassembled, the circuit is disconnected, and it can be determined that the tray of the storage layer is disassembled. For another example, the position of each storage layer where the pallets are installed can be pre-set, and the size of each pallet is a fixed value. The associated camera can take images when all pallets are installed, and when the pallets are removed, and calibrate the pallet area in each state. Then the camera can determine which pallets are removed based on the size and position of the pallet images in the captured images. For another example, since the camera is installed on the upper storage layer, the automatic recognition function can be determined based on whether the image signal of each camera is input to the processor, combined with the detection sensor set at the connection between the body and the pallet, to improve the accuracy.

[0044] S102. Re-determine the associated cameras of the respective remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information.

[0045] In the embodiment of the present invention, for any storage layer, the associated camera of the storage layer is arranged at the bottom of the target storage layer adjacent to the storage layer and above the storage layer. Figure 1a , the associated camera b of storage layer 2 is set at the bottom of storage layer 1; the associated camera c of storage layer 3 is set at the bottom of detachable storage layer 2; wherein storage layers 2 and 3 are detachable storage layers. If storage layer 2 is disassembled, the associated camera c set at the bottom of storage layer 2 for collecting images in storage layer 3 is removed. At this time, the associated camera b is used to collect images in storage layer 3, so that the detection distance of the associated camera b becomes larger; wherein, the detection distance refers to the distance from the associated camera to the bottom of the storage layer it collects. It can be seen that the disassembly of the storage layer causes the association relationship between some cameras and the storage layer to change, and the detection distance of some cameras also changes. In this way, after determining that one or more storage layers are disassembled, it is necessary to re-determine the associated cameras of the remaining storage layers of the delivery robot and the detection distances of the associated cameras.

[0046] In an optional implementation, the disassembled storage layer and the removed associated camera are determined based on the storage layer change information, and then the associated cameras of the current remaining storage layers and the detection distances of the associated cameras can be determined by automatic storage layer identification. In addition, since the number of detachable storage layers on the delivery robot is limited, it is also possible to use statistical methods to count the associated cameras and the detection distances of the associated cameras of the remaining storage layers after any storage layer is disassembled in advance, and save them as statistical results; then, after determining that one or more storage layers are disassembled based on the storage layer change information, the associated cameras of the current remaining storage layers and the detection distances of the associated cameras can be directly determined from the statistical results.

[0047] S103: adjusting the storage layer image captured by the associated camera according to the detection distance of the associated camera.

[0048] In the embodiment of the present invention, disassembling the storage layer will cause the detection distance of a certain associated camera to increase. For example, when storage layer 2 is disassembled, the original detection distance of associated camera b is equal to the height of storage layer 2. After storage layer 2 is disassembled, the detection distance of associated camera b is equal to the height of storage layer 2 plus the height of storage layer 3. The increase in the detection distance of the associated camera will cause the size of the object in the image it collects to decrease. At this time, if the image it collects is still directly used for object detection, the accuracy of object detection will decrease. Therefore, in order to ensure the accuracy of object detection, the image of the storage layer taken by the associated camera can be adjusted according to the detection distance of the associated camera, for example, the image of the storage layer can be enlarged to achieve the effect of enlarging the features of the object; and then the enlarged image can be used for object detection later, which can improve the accuracy of object detection.

[0049] It should be noted that if the detection distance of the associated camera does not change, there is no need to adjust the storage layer image captured by it.

[0050] S104: Detecting objects in the storage layer based on the adjusted storage layer image.

[0051] In the embodiment of the present invention, since the delivery robot is provided with multiple storage layers, the delivery robot can deliver different items to different users at the same time. Specifically, the delivery robot can plan the delivery path according to different delivery destinations of the items, and then deliver the items in sequence according to the delivery path; wherein, the name and destination of each delivery item can be displayed in the display device of the delivery robot. Since the delivery robot may deliver items to different users at the same time, in addition to detecting whether the user has completed the pickup, it is also necessary to detect whether the user is picking up the goods at the correct storage layer. On this basis, when the delivery robot arrives at a target delivery point (i.e., the delivery destination), based on the adjusted storage layer image and combined with the pre-trained item detection model, it is determined whether the items in the storage layer have been taken away, that is, whether the captured image includes the items. If there are no items in the image, it means that the items have been taken away, and there is no need to manually determine the meal pickup; and before the robot continues to deliver other items, it can also determine whether there is a situation where the item is taken wrongly according to the current location of the delivery robot and the target delivery point of the delivered item in the storage layer; if there is a situation where the wrong item is taken, a wrong pickup reminder is made, such as a voice prompting the customer to take the wrong item.

[0052] The technical solution of the embodiment of the present invention is to set a detachable storage layer on the robot. By disassembling the storage layer, the problem that the robot cannot deliver items that are higher than the storage layer is solved, and higher objects such as cakes and beer can be delivered; and after adjusting the storage layer, the associated cameras corresponding to the remaining storage layers and the latest detection distance of the associated cameras are promptly determined, and the captured image is adjusted according to the detection distance to magnify the object features, thereby ensuring the accuracy of object detection based on the adjusted image. At this time, the plate detection model can still recognize with high precision, supporting customers to pick up meals by themselves without clicking to confirm the pick-up, which improves the user experience and provides merchants with a variety of delivery options.

[0053] Embodiment 2

[0054] Figure 2 This is a flow chart of an object detection method provided in Example 2 of the present invention. This embodiment is optimized based on the above embodiment. Figure 2 The method flow includes the following steps:

[0055] S201. Obtain the storage layer change information of the delivery robot.

[0056] S202. Re-determine the associated cameras of the remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information and in combination with a preset mapping relationship.

[0057] In this embodiment, since the number of detachable storage layers on the delivery robot is limited, a mapping relationship can be pre-constructed based on manual statistics; wherein the mapping relationship may include the detection distance between each associated camera and each storage layer, for example, Figure 1a In the mapping relationship, camera b is associated, and the detection distances of the associated camera b and the storage layer 2, storage layer 3 and storage layer 4 are saved respectively; the mapping relationship may also include the association relationship between the remaining storage layers and the remaining cameras after any storage layer is removed; illustratively, after the storage layer 2 is removed, the storage layer 1 is associated with camera a, and the storage layer 3 is associated with camera b; after the storage layer 3 is removed, the storage layer 1 is associated with camera a, the storage layer 2 is associated with camera b, and the storage layer 4 is associated with camera c; after the storage layer 2 and the storage layer 3 are removed at the same time, the storage layer 1 is associated with camera a, and the storage layer 4 is associated with camera b. In some embodiments, considering that the bottom layer is used less frequently, a camera may not be set at the bottom of the tray of the storage layer 3 to save costs. That is, when the tray of the storage layer 3 is not removed, the storage layer 4 does not have the item detection function, but when the tray of the storage layer 3 is removed, the storage layer 4 may be provided with the item detection function.

[0058] On the basis of the predetermined mapping relationship, as long as the disassembled storage layer is determined according to the storage layer change information, the associated cameras of the current remaining storage layers and the detection distances of the associated cameras can be determined from the mapping relationship.

[0059] S203: adjusting the storage layer image captured by the associated camera according to the detection distance of the associated camera.

[0060] The disassembly and assembly information of the storage layer will be given to the app, and then to the algorithm to determine whether the image needs to be processed. Suppose that the storage layer 2 is disassembled, the camera is only [a, b], and the storage layer is [1, 3, 4]. At this time, the algorithm knows that the second layer is disassembled, and the camera a takes a picture of the storage layer 1, and the camera b takes a picture of the storage layer 3. At this time, the algorithm knows to process the picture and perform the resize operation. Suppose that the storage layer 3 is disassembled, the camera does not change to [a, b, c], and the storage layer becomes [1, 2, 4]. The camera a takes a picture of the dining layer 1, the camera b takes a picture of the dining layer 2, and the camera c takes a picture of the dining layer 4. At this time, the algorithm knows to process the image and perform the resize operation.

[0061] S204: Detect the objects in the storage layer based on the adjusted storage layer image.

[0062] In this embodiment, by presetting a mapping relationship, the associated cameras of the remaining storage layers and the detection distances of the associated cameras can be directly determined based on the mapping relationship and storage layer change information. No further analysis by the delivery robot is required, thus saving computing power and improving efficiency.

[0063] Embodiment 3

[0064] Figure 3 This is a flow chart of an object detection method provided by Embodiment 3 of the present invention. Figure 3 The method flow includes the following steps:

[0065] S301. Obtain the storage layer change information of the delivery robot.

[0066] In an embodiment of the present invention, if the mapping relationship involved in the above embodiment is not pre-constructed, the associated cameras of the remaining storage layers of the delivery robot and the detection distances of the associated cameras can be re-determined based on the automatic identification function of the storage layers of the delivery robot and the storage layer change information. The specific implementation process can be found in steps S302-S305.

[0067] S302: Determine the first target storage layer that is currently disassembled according to the storage layer change information.

[0068] In this embodiment, the storage layer change information of the delivery robot may optionally be the relevant information of the storage layer that has been disassembled by the delivery robot, such as the identification information of the disassembled storage layer and the information of the associated camera of the disassembled storage layer. In this way, the first target storage layer that is currently disassembled can be determined directly based on the identification information of the disassembled storage layer. It should be noted that since there are multiple detachable storage layers on the delivery robot, the disassembled storage layer may be one or more. Therefore, it is also necessary to determine the first target storage layer based on the number of disassembled storage layers to ensure the accuracy of subsequent identification. Optionally, if it is determined that the number of disassembled storage layers is one according to the identification information of the disassembled storage layer in the storage layer change information, the disassembled storage layer is used as the first target storage layer; if it is determined that the number of disassembled storage layers is at least two according to the identification information of the disassembled storage layer in the storage layer change information, the positional relationship of the disassembled storage layers is determined, wherein the positional relationship may be the distribution relationship of the storage layers in the vertical direction; based on the positional relationship, the storage layer at the top position among the disassembled storage layers is used as the first target storage layer; for example, if storage layer 2 and storage layer 3 are disassembled at the same time, since storage layer 2 is distributed above storage layer 3, storage layer 2 is used as the first target storage layer.

[0069] S303: Determine a second target storage layer that currently exists on the delivery robot and is located below the first target storage layer and adjacent to the first target storage layer.

[0070] In this embodiment, the delivery robot can automatically identify the currently remaining storage layers, and then according to the distribution relationship of each storage layer in the vertical direction, can quickly determine the currently existing second target storage layer located below the first target storage layer and adjacent to the first target storage layer.

[0071] S304: Re-establish an association relationship between the first associated camera corresponding to the first target storage layer and the second target storage layer.

[0072] Optionally, the association between the first target storage layer and the first associated camera is released; the association between the second associated camera and the second target storage layer is released; wherein the second associated camera is located at the bottom of the first target storage layer and is removed from the robot as the first target storage layer is disassembled; the first associated camera corresponding to the first target storage layer is re-associated with the second target storage layer, so that the first associated camera is used to capture images of the second target storage layer.

[0073] S305: Re-determine the detection distance of the first associated camera according to the height attribute of the storage layer and the number of the disassembled storage layers.

[0074] Among them, the height attribute is an optional layer height. In this embodiment, the layer height of each storage layer on the delivery robot is the same as h. On this basis, the number of disassembled storage layers is n, and the detection distance of the first associated camera is L = h*(n+1). It should be noted that if the layer heights of the storage layers are different, the layer height of the storage layer initially associated with the first associated camera is added to the layer height of the disassembled storage layer to obtain the final detection distance.

[0075] S306: Adjust the storage layer image captured by the first associated camera according to the detection distance of the first associated camera.

[0076] Optionally, an image adjustment coefficient is determined based on re-determining the detection distance of the first associated camera; wherein the image adjustment coefficient can be optionally an image magnification factor; different detection distances correspond to different image magnification factors; and the storage layer image taken by the first associated camera is adjusted based on the image adjustment coefficient.

[0077] S307: Detect the objects in the storage layer based on the adjusted storage layer image.

[0078] The solution of this embodiment can automatically identify the associated cameras corresponding to the remaining storage layers and the detection distances of the associated cameras, thereby providing a guarantee for subsequent accurate detection of items.

[0079] Embodiment 4

[0080] Figure 4 This is a flow chart of an object detection method provided by Embodiment 4 of the present invention. Figure 4 The method flow includes the following steps:

[0081] S401. Obtain the storage layer change information of the delivery robot.

[0082] S402. Re-determine the associated cameras of the respective remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information.

[0083] S403: Determine an image adjustment coefficient according to the detection distance of the associated camera.

[0084] The image adjustment coefficient may be an image magnification factor; different detection distances correspond to different image magnification factors.

[0085] S404: adjusting the storage layer image captured by the associated camera according to the image adjustment coefficient.

[0086] In an optional implementation, the storage layer image taken by the associated camera is enlarged according to the magnification factor corresponding to the detection distance, so as to achieve the effect of enlarging the features of the object in the image, thereby ensuring the accuracy of object detection based on the enlarged image.

[0087] In another optional implementation scheme, the storage layer image captured by the associated camera is adjusted according to the image adjustment coefficient, including: determining the pallet vertex coordinates of the corresponding associated storage layer from the storage layer image captured by the associated camera, and determining the pallet area based on the pallet vertex coordinates; wherein, because the installation position, viewing angle and layer height between each storage layer of the associated camera are fixed, the pallet vertex coordinates can be determined by a pre-calibrated method. Determining the pallet vertex coordinates by a pre-calibrated method can avoid misjudgment caused by objects on the pallet blocking the edge of the pallet. The pallet vertex coordinates can also be obtained by the associated camera according to image recognition to identify the edge contour of the pallet. In addition, the target recognition algorithm can also be used to determine the pallet vertex coordinates and the pallet area; according to the pallet area, the pallet area image is intercepted from the storage layer image captured by the associated camera to eliminate the influence of interference objects in the image on the object detection, wherein the interference objects in the image are collected because the camera detection distance becomes larger; according to the image adjustment coefficient, the pallet area image is adjusted. It should be noted that when performing object detection, the input image needs to be tested in a pre-trained object detection model, so it is only necessary to enlarge the size of the tray area image to be consistent with the sample image size used for model training.

[0088] First redetermine the tray area, then adjust the image of the tray area, can improve the accuracy of object detection, and the adjusted image can be input into the pre-made object detection model, the object detection model does not need to be adjusted according to different detection distances, improves the versatility of the model, and reduces the difficulty of training. For example, if the image size of the tray area is 800*600 during model training, the adjusted image is also adjusted to 800*600 to keep consistent.

[0089] S405: Detect the objects in the storage layer based on the adjusted storage layer image.

[0090] In the present embodiment, the image of the storage layer taken by the associated camera is enlarged, so that the features of the object can be enlarged, so that when the object is identified based on the enlarged image, the accuracy of the object identification can be guaranteed.

[0091] Embodiment 5

[0092] Figure 5 This is a flow chart of an object detection method provided in Embodiment 5 of the present invention. This embodiment is also applicable to the case where, after the robot that has completed the disassembly of the storage layer has delivered the object, part or all of the disassembled storage layer is reinstalled on the delivery robot, see Figure 5 , the method logic includes the following:

[0093] S501. Obtain the storage layer change information of the delivery robot.

[0094] The storage layer change information may also include identification information of the newly installed storage layer. Accordingly, the cameras associated with the remaining storage layers of the delivery robot are re-determined based on the storage layer change information, including steps S502-S504.

[0095] S502: If it is determined according to the identification information of the newly installed storage layer that the number of the newly installed storage layer is one, the newly installed storage layer is used as the third target storage layer.

[0096] S503: Determine a fourth target storage layer that currently exists in the delivery robot and is located below the third target storage layer and adjacent to the third target storage layer.

[0097] S504: Establish an association relationship between the third associated camera associated with the fourth target storage layer and the third target storage layer, so that the third associated camera is used to capture an image of the third target storage layer.

[0098] S505: Establish an association relationship between a fourth associated camera at the bottom of the third target storage layer and the fourth target storage layer, so that the fourth associated camera is used to capture an image of the fourth target storage layer.

[0099] S506: Identify the items in the third storage layer and the fourth storage layer based on the image of the third storage layer and the image of the fourth storage layer respectively.

[0100] In this embodiment, when a new storage layer is installed, by adjusting the association relationship, it can be ensured that the camera can correctly capture the image of the associated storage layer, thereby ensuring the accuracy of object recognition on each storage layer.

[0101] In this embodiment, if the number of the newly installed storage layers is at least two, the association relationship between the storage layers and the cameras is re-established according to the currently existing storage layers and cameras.

[0102] Embodiment 6

[0103] Figure 6 This is a schematic diagram of the structure of an object detection device provided in the fourth embodiment of the present invention. This embodiment is applicable to the scene where robots are used to deliver objects, and is typically applicable to the scene where robots are used to deliver meals in restaurants. Figure 6 As shown, the device is configured on a delivery robot, the delivery robot includes multiple storage layers including at least one detachable storage layer, each storage layer is associated with a camera for collecting storage layer images; the device specifically includes:

[0104] The storage layer change information acquisition module 601 is used to acquire the storage layer change information of the delivery robot;

[0105] The association relationship determination module 602 is used to re-determine the associated cameras of the respective remaining storage layers of the delivery robot and the detection distance of the associated cameras according to the storage layer change information;

[0106] An image adjustment module 603, used to adjust the storage layer image taken by the associated camera according to the detection distance of the associated camera;

[0107] The detection module 604 is used to detect the objects in the storage layer based on the adjusted storage layer image.

[0108] Based on the above embodiment, optionally, the association relationship determination module includes:

[0109] The first determination unit is used to re-determine the associated cameras of the respective currently remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information and in combination with a preset mapping relationship.

[0110] Based on the above embodiment, optionally, the association relationship determination module further includes:

[0111] A second determining unit, configured to determine a first target storage layer that is currently being disassembled according to the storage layer change information;

[0112] A third determining unit is used to determine a second target storage layer currently existing on the delivery robot and located below the first target storage layer and adjacent to the first target storage layer;

[0113] an association relationship determination unit, configured to re-establish an association relationship between the first association camera corresponding to the first target storage layer and the second target storage layer;

[0114] The detection distance determination unit is used to re-determine the detection distance of the first associated camera according to the height attribute of the storage layer and the number of the storage layers that have been disassembled.

[0115] Based on the above embodiment, optionally, the second determining unit is further configured to:

[0116] If it is determined that the number of the disassembled storage layers is one according to the identification information of the disassembled storage layer in the storage layer change information, the disassembled storage layer is used as the first target storage layer;

[0117] If it is determined that the number of the disassembled storage layers is at least two according to the identification information of the disassembled storage layers in the storage layer change information, the positional relationship of the disassembled storage layers is determined; and according to the positional relationship, the storage layer at the top position among the disassembled storage layers is used as the first target storage layer.

[0118] Based on the above embodiment, optionally, the image adjustment module includes:

[0119] A coefficient determination unit, used to determine an image adjustment coefficient according to a detection distance of the associated camera;

[0120] The adjustment unit is used to adjust the storage layer image taken by the associated camera according to the image adjustment coefficient.

[0121] Based on the above embodiment, optionally, the adjustment unit is further used for:

[0122] Determine the pallet vertex coordinates corresponding to the associated storage layer from the storage layer image taken by the associated camera, and determine the pallet area based on the pallet vertex coordinates;

[0123] According to the pallet area, capturing a pallet area image from the storage layer image taken by the associated camera;

[0124] The tray area image is adjusted according to the image adjustment coefficient.

[0125] Based on the above embodiment, optionally, the storage layer at the top layer of the delivery robot is a non-removable storage layer; and the non-removable storage layer is provided with a stereoscopic vision sensor for collecting point cloud data of the surrounding environment of the delivery robot, so that the delivery robot can avoid obstacles according to the point cloud data of the surrounding environment.

[0126] The object detection device provided in the embodiment of the present invention can execute the object detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0127] Embodiment 7

[0128] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0129] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0131] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as executing an object detection method.

[0132] In some embodiments, the object detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the object detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the object detection method in any other appropriate manner (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable article detection device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0138] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting an object, characterized in that: Applied to a delivery robot, the delivery robot includes multiple storage layers including at least one detachable storage layer, each storage layer is associated with a camera for collecting images of the storage layer; the method includes: Obtaining storage layer change information of the delivery robot; Re-determining the associated cameras of the respective remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information; According to the detection distance of the associated camera, adjust the storage layer image taken by the associated camera; Based on the adjusted storage layer image, the objects in the storage layer are detected.

2. The method according to claim 1, characterized in that Re-determining the associated cameras of the respective remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information, including: According to the storage layer change information and in combination with a preset mapping relationship, the associated cameras of the respective currently remaining storage layers of the delivery robot and the detection distances of the associated cameras are re-determined.

3. The method according to claim 1, characterized in that Re-determining the associated cameras of the respective remaining storage layers of the delivery robot and the detection distances of the associated cameras according to the storage layer change information, including: Determining a first target storage layer that is currently disassembled according to the storage layer change information; Determine a second target storage layer currently existing on the delivery robot and located below the first target storage layer and adjacent to the first target storage layer; Re-establishing an association relationship between the first associated camera corresponding to the first target storage layer and the second target storage layer; The detection distance of the first associated camera is re-determined according to the height attribute of the storage layer and the number of the disassembled storage layers.

4. The method according to claim 3, characterized in that Determining the first target storage layer currently being disassembled according to the storage layer change information includes: If it is determined that the number of the disassembled storage layer is one according to the identification information of the disassembled storage layer in the storage layer change information, the disassembled storage layer is used as the first target storage layer; If it is determined that the number of the disassembled storage layers is at least two according to the identification information of the disassembled storage layers in the storage layer change information, the positional relationship of the disassembled storage layers is determined; and according to the positional relationship, the storage layer at the top position among the disassembled storage layers is used as the first target storage layer.

5. The method according to claim 1, characterized in that According to the detection distance of the associated camera, adjusting the storage layer image captured by the associated camera includes: Determining an image adjustment coefficient according to the detection distance of the associated camera; The storage layer image captured by the associated camera is adjusted according to the image adjustment coefficient.

6. The method according to claim 5, characterized in that Adjusting the storage layer image captured by the associated camera according to the image adjustment coefficient includes: Determine the pallet vertex coordinates corresponding to the associated storage layer from the storage layer image taken by the associated camera, and determine the pallet area based on the pallet vertex coordinates; According to the pallet area, capturing a pallet area image from the storage layer image taken by the associated camera; The tray area image is adjusted according to the image adjustment coefficient.

7. The method according to claim 1, characterized in that The storage layer at the top layer of the delivery robot is a non-detachable storage layer; and the non-detachable storage layer is provided with a stereoscopic vision sensor for collecting point cloud data of the surrounding environment of the delivery robot, so that the delivery robot can avoid obstacles according to the point cloud data of the surrounding environment.

8. An object detection device, characterized in that: The device is configured on a delivery robot, wherein the delivery robot includes a plurality of storage layers including at least one detachable storage layer, and each storage layer is associated with a camera for collecting images of the storage layer; the device includes: A storage layer change information acquisition module, used to acquire storage layer change information of the delivery robot; An association relationship determination module, used to re-determine the associated cameras of the respective remaining storage layers of the delivery robot and the detection distance of the associated cameras according to the storage layer change information; An image adjustment module, used for adjusting the storage layer image taken by the associated camera according to the detection distance of the associated camera; The detection module is used to detect the objects in the storage layer based on the adjusted storage layer image.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

Images