Intelligent control method for sorting device based on depth camera and three-axis movement mechanism

Abstract

The invention is suitable for the technical field of image processing, and provides a sorting device intelligent control method based on a depth camera and a three-axis motion mechanism, and the method comprises the steps: firstly, based on the depth camera, rapidly obtaining real-time depth image information, and then according to the real-time depth image information and a preset target detection algorithm, obtaining a target detection result; the target category information and the first real-time position information of the to-be-detected object are effectively generated, then the second real-time position information of the target storage basket is accurately determined based on the target category information, and finally the sorting instruction information is effectively generated according to the first real-time position information and the second real-time position information. According to the method, the image data collected by the depth camera in real time can be rapidly processed, then the intelligent control system completes object recognition, positioning and motion trail planning in a very short time through a high-performance processor and an optimized algorithm, object sorting work is completed in combination with three-dimensional space data, and the working efficiency is improved. And the grabbing success rate is greatly improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an intelligent control method for a sorting device based on a depth camera and a three-axis motion mechanism. Background Technology

[0002] Sorting robots, with their highly efficient automated operation capabilities, have significantly improved sorting efficiency and reduced labor intensity and labor costs, thus gradually replacing traditional manual sorting operations in industrial production.

[0003] Currently, most traditional sorting robots rely on two-dimensional cameras to acquire planar images of objects. However, due to the lack of perception of three-dimensional spatial data, the robots have difficulty accurately determining the specific location of objects, which significantly limits the accuracy and stability of sorting and results in a low success rate of grasping. Further improvements are needed. Summary of the Invention

[0004] Based on this, embodiments of this application provide an intelligent control method for a sorting device based on a depth camera and a three-axis motion mechanism to solve the problem of low grasping success rate in the prior art.

[0005] In a first aspect, embodiments of this application provide an intelligent control method for a sorting device based on a depth camera and a three-axis motion mechanism, the method comprising:

[0006] Based on a preset depth camera, acquire real-time depth image information; Based on the real-time depth image information and the preset target detection algorithm, target category information and first real-time position information of the object to be detected are generated. Based on the target category information, determine the second real-time location information of the target storage basket; Based on the first real-time location information and the second real-time location information, sorting instruction information is generated, wherein the sorting instruction information is used to instruct the target robot to grasp the object to be detected and move the object to be detected from the first real-time location information to the second real-time location information.

[0007] Compared with the prior art, the beneficial effects are as follows: The intelligent control method for sorting devices based on depth cameras and three-axis motion mechanisms provided in this application embodiment allows the terminal device to first quickly acquire real-time depth image information based on the depth camera, and then effectively generate target category information and first real-time position information of the object to be detected based on the real-time depth image information and the preset target detection algorithm. Then, based on the target category information, the second real-time position information of the target storage basket is accurately determined. Finally, based on the first real-time position information and the second real-time position information, sorting instruction information is effectively generated, thereby realizing the sorting of objects by combining three-dimensional spatial data, effectively improving the grasping success rate, and solving the problem of low grasping success rate to a certain extent.

[0008] Secondly, embodiments of this application provide an intelligent control system for a sorting device based on a depth camera and a three-axis motion mechanism, the system comprising: Real-time depth image information acquisition module: used to acquire real-time depth image information based on a preset depth camera; Target category information generation module: used to generate target category information and first real-time position information of the object to be detected based on the real-time depth image information and the preset target detection algorithm; The second real-time location information determination module is used to determine the second real-time location information of the target storage basket based on the target category information. Sorting instruction information generation module: used to generate sorting instruction information based on the first real-time location information and the second real-time location information, wherein the sorting instruction information is used to instruct the target robot to grab the object to be detected and move the object to be detected from the first real-time location information to the second real-time location information.

[0009] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0010] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0011] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0013] Figure 1 This is a flowchart illustrating an intelligent control method for a sorting device provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the process before step S100 in the intelligent control method for a sorting device provided in an embodiment of this application. Figure 3 This is a flowchart illustrating the process after step S104 in the intelligent control method for a sorting device provided in an embodiment of this application. Figure 4 This is a flowchart illustrating the process after step S200 in the intelligent control method for a sorting device provided in an embodiment of this application. Figure 5 This is a flowchart illustrating the process after step S400 in the intelligent control method for a sorting device provided in an embodiment of this application. Figure 6 This is a block diagram of the intelligent control system for a sorting device provided in one embodiment of this application; Figure 7 This is a schematic diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0016] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0017] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0018] Please see Figure 1 , Figure 1 This is a flowchart illustrating the intelligent control method for a sorting device based on a depth camera and a three-axis motion mechanism provided in this embodiment. In this embodiment, the executing entity of the intelligent control method for the sorting device is a terminal device. It is understood that the types of terminal devices include, but are not limited to, mobile phones, tablets, laptops, Ultra-Mobile Personal Computers (UMPCs), netbooks, Personal Digital Assistants (PDAs), etc. This embodiment does not impose any restrictions on the specific type of terminal device.

[0019] Please see Figure 1 The intelligent control method for the sorting device provided in this application includes, but is not limited to, the following steps: In the S100, real-time depth image information is acquired based on a preset depth camera.

[0020] Without loss of generality, a sorting device, or sorting robot, can be an automated system integrating advanced sensing, precise control, and efficient execution functions. In one possible implementation, the sorting device can mainly consist of a depth camera, an image processing unit, a controller, a three-axis motion mechanism (such as a motor and transmission device), related sensors, and a communication module. In another possible implementation, maintenance personnel can pre-equip the depth camera with optical compensation functions such as automatic aperture, automatic exposure, and automatic white balance.

[0021] In one possible implementation, since the depth camera is the key sensing component of the entire system, it can be pre-mounted in a stable bracket or housing to avoid mechanical impact and vibration.

[0022] Specifically, the terminal device can acquire real-time depth image information based on a preset depth camera. The real-time depth image information is used to describe the images obtained by taking real-time pictures of the sorting scene using the depth camera.

[0023] For some possible implementations, please refer to [link / reference needed] to improve recognition accuracy. Figure 2 Before step S100, the method further includes, but is not limited to, the following steps: In S101, in response to the self-test command, multiple test image information are continuously acquired based on the depth camera.

[0024] Specifically, the terminal device can first respond to the self-test command and continuously acquire multiple test image information based on the depth camera. The self-test command is used to instruct the device to determine whether the depth camera is covered by contaminants such as oil. The test image information is used to describe the images taken to test whether there are contaminants such as oil on the camera lens.

[0025] Without loss of generality, the shooting angles corresponding to multiple test image information are all different, and the specific number of test image information can be customized. For example, the terminal device can first control the depth camera to shoot a test image information, then control the depth camera to rotate 90 degrees clockwise in the horizontal direction to shoot another test image information, and then control the depth camera to continue to rotate 90 degrees clockwise in the horizontal direction to shoot another test image information, for a total of three test image information.

[0026] In S102, for each test image information: based on the preset edge extraction algorithm, multiple contour information to be detected is generated.

[0027] Specifically, after the terminal device continuously acquires multiple test image information, the terminal device can perform the following processing on each test image information: based on a preset edge extraction algorithm, perform edge contour extraction processing on the test image information to generate multiple contour information to be detected, wherein the contour information to be detected is used to describe the edge contour extracted by the edge contour extraction processing on the test image information.

[0028] In S103, it is determined whether there is at least one detectable contour information with the same location and size among the multiple test image information.

[0029] Specifically, when there are contaminants such as oil on the lens of a depth camera, the contour information to be detected will have an oily contour. Therefore, after the terminal device generates multiple contour information to be detected, the terminal device can determine whether there is at least one contour information to be detected with the same position and size among the multiple test image information.

[0030] In S104, if at least one detectable contour information with the same location and size exists in multiple test image information, an oil stain warning message is generated.

[0031] Specifically, if at least one detection contour information with the same location and size exists in multiple test image information, it indicates that there are contaminants such as oil on the depth camera lens. Therefore, the terminal device can generate an oil contamination warning message, which is used to indicate that there are contaminants such as oil on the depth camera lens. When there is oil on the depth camera lens, it seriously affects the detection accuracy.

[0032] For a wider range of possible implementations that are suitable for various applications and improve detection accuracy, please refer to [link / reference]. Figure 3 After step S104, the method further includes, but is not limited to, the following steps: In S105, it is determined whether oil stain removal information has been received.

[0033] Specifically, after the terminal device generates an oil stain warning message, the terminal device can determine whether it has received an oil stain removal message. The oil stain removal message is sent by the user to the terminal device through their terminal. After the user has cleaned the oil stains on the depth camera lens, the user can send the oil stain removal message to the terminal device.

[0034] In S106, if oil stain removal information is received, the process continues to execute based on a preset depth camera to acquire real-time depth image information.

[0035] Specifically, if the terminal device receives the oil stain removal information, the terminal device can continue to perform the above step S100.

[0036] In S107, if no oil stain removal information is received, an oil stain outline removal instruction is generated based on the oil stain prompt information.

[0037] Specifically, if the terminal device does not receive oil stain removal information, it can generate an oil stain contour removal instruction based on the oil stain prompt information, thereby reducing the adverse effect of oil stain contours on image detection. The oil stain contour removal instruction is used to instruct the removal of oil stain contours in the real-time depth image information. The oil stain contours are the same contour information to be detected in multiple test image information with the same position and size.

[0038] In S200, target category information and first real-time position information of the object to be detected are generated based on real-time depth image information and a preset target detection algorithm.

[0039] Specifically, after the terminal device acquires real-time depth image information, the terminal device can use a target detection algorithm to perform target detection processing on the real-time depth image information, identify the target category information corresponding to the object to be detected in the real-time depth image information, and further determine the first real-time location information. The first real-time location information is used to describe the real-time location of the object to be detected, which can be a ceramic cup, a plastic cup, or a metal cup.

[0040] For example, since each pixel value in a depth image directly encodes the physical distance from the camera to the corresponding point in the scene, the terminal device can first perform a core coordinate transformation based on the camera's intrinsic parameters (such as focal length and optical center). This transformation converts the "row and column coordinates + depth value" of each pixel into three-dimensional point coordinates in the camera coordinate system through geometric calculations, thereby generating point cloud data representing the surface sampling of the object. Then, object detection algorithms are used to separate independent objects from the point cloud data. For example, depth mutations can be used to identify boundaries, or clustering algorithms can be used to group spatially adjacent points into the same group. Combined with the recognition results of the color image, semantic labels such as "ceramic cup," "plastic cup," or "metal cup" are assigned to the corresponding point sets. Once the object is separated, the basic position is obtained by calculating the geometric centroid of the point set, or a smaller three-dimensional bounding box that can completely enclose the object is constructed to obtain more accurate center coordinates, thereby determining its specific location.

[0041] In some possible implementations, to supplement and calibrate depth camera data by incorporating other sensors, please refer to [link to relevant documentation]. Figure 4 After step S200, the method further includes, but is not limited to, the following steps: In S201, the interval distance information of the object to be detected is obtained based on the preset laser rangefinder and contact displacement sensor.

[0042] Specifically, the terminal device can quickly acquire the distance information of the object to be detected based on the preset laser rangefinder and contact displacement sensor. The distance information is used to describe the distance between the object to be detected and the depth camera, calculated by the laser rangefinder and contact displacement sensor.

[0043] In S202, based on the interval distance information, the third real-time position information of the object to be detected is generated.

[0044] Specifically, after the terminal device acquires the interval distance information, the terminal device can generate the third real-time location information of the object to be detected based on the interval distance information. The third real-time location information is used to describe the real-time location of the object to be detected determined using the interval distance information.

[0045] In S203, it is determined whether the first real-time location information is the same as the third real-time location information.

[0046] Specifically, after the terminal device generates the third real-time location information, the terminal device can determine whether the first real-time location information is the same as the third real-time location information.

[0047] In S204, if the first real-time location information is not the same as the third real-time location information, then the average distance information is generated based on the first real-time location information and the third real-time location information.

[0048] Specifically, if the first real-time location information is not the same as the third real-time location information, the terminal device can generate the average distance information based on the average value calculated by adding the first real-time location information to the third real-time location information.

[0049] In S205, the fourth real-time position information of the object to be detected is generated based on the average spacing information.

[0050] Specifically, after the terminal device generates the average spacing information, the terminal device can generate the fourth real-time position information of the object to be detected based on the average spacing information. The fourth real-time position information is used to describe the real-time position of the object to be detected determined using the average spacing information.

[0051] In S206, the first real-time location information is modified to the fourth real-time location information.

[0052] Specifically, after the terminal device generates the fourth real-time location information, the terminal device can modify the first real-time location information to the fourth real-time location information, and then execute the subsequent step S300 again.

[0053] In one possible implementation, the terminal device can also use more accurate distance information provided by auxiliary sensors, combined with data fusion algorithms (such as Kalman filtering), to fuse measurement data from different sensors, thereby improving the accuracy of object position and shape determination.

[0054] In S300, the second real-time location information of the target storage basket is determined based on the target category information.

[0055] Specifically, after the terminal device determines the second real-time location information, the terminal device can determine the storage basket specifically for loading objects of that category based on the target category information, thereby effectively determining the second real-time location information of the target storage basket. The target storage basket is used to describe the storage basket specifically for loading objects of that category, and the second real-time location information is used to describe the real-time location of the target storage basket.

[0056] In S400, sorting instruction information is generated based on the first real-time location information and the second real-time location information.

[0057] Specifically, after the terminal device determines the second real-time location information, the terminal device can effectively generate sorting instruction information based on the first real-time location information and the second real-time location information. The sorting instruction information is used to instruct the target robot to grab the object to be detected and move the object to be detected from the first real-time location information to the second real-time location information.

[0058] In some possible implementations, to help operations and maintenance personnel understand the specific sorting status, please refer to [link / reference]. Figure 5 After step S400, the method further includes, but is not limited to, the following steps: In S500, the total quantity information and the sorted quantity information corresponding to each target category information are obtained.

[0059] Specifically, after the terminal device generates sorting instruction information, the terminal device can obtain the total quantity information and sorted quantity information corresponding to each target category information. The total quantity information is used to describe the total number of objects to be detected in the sorting scenario, and the sorted quantity information is used to describe the number of objects to be detected that have been moved into the storage basket.

[0060] In S510, sorting ratio information is generated based on the total quantity information and the sorted quantity information.

[0061] Specifically, after the terminal device obtains the total quantity information and the sorted quantity information, it can generate the sorted ratio information based on the quotient of the sorted quantity information divided by the total quantity information.

[0062] In S520, sorting task set information is generated based on the sorting ratio information corresponding to each target category information.

[0063] Specifically, after the terminal device generates the sorted ratio information, it can generate sorting task set information based on the sorted ratio information corresponding to each target category. The sorting task set information describes a data set that integrates the sorted ratio information corresponding to each target category.

[0064] In S530, sorting task set information is sent to the designated terminal.

[0065] Specifically, after the terminal device generates the sorting task set information, the terminal device can send the sorting task set information to a designated terminal, where the designated terminal can be the user's terminal.

[0066] In one possible implementation, maintenance personnel can further reduce the inertia and frictional resistance of the target robotic arm during mechanical movement by optimizing the motor control parameters and the design of the transmission mechanism. This allows the target robotic arm to complete the movement and grasping action from the initial position to the target position in a short time. The implementation principle of the intelligent control method for a sorting device based on a depth camera and a three-axis motion mechanism in this application embodiment is as follows: The terminal device can first quickly acquire real-time depth image information based on the depth camera. Then, based on the real-time depth image information and a preset target detection algorithm, it effectively generates target category information and first real-time position information of the object to be detected. This can accurately identify and locate the specific position of each object, ensuring that the target robotic arm can accurately grasp it in the subsequent process, greatly reducing missorting caused by inaccurate positioning. Then, based on the target category information, it accurately determines the second real-time position information of the target storage basket. Finally, based on the first and second real-time position information, it effectively generates sorting instruction information. This enables the image data acquired by the depth camera to be processed quickly. The intelligent control system uses a high-performance processor and optimized algorithms to complete the object identification, positioning, and motion trajectory planning in a very short time. Combined with three-dimensional spatial data, it realizes the sorting of objects, effectively improving the grasping success rate.

[0067] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0068] Embodiments of this application also provide an intelligent control system for a sorting device based on a depth camera and a three-axis motion mechanism. For ease of explanation, only the parts relevant to this application are shown, such as... Figure 6 As shown, the system 60 includes: Real-time depth image information acquisition module 61: used to acquire real-time depth image information based on a preset depth camera; Target category information generation module 62: used to generate target category information and first real-time position information of the object to be detected based on real-time depth image information and preset target detection algorithm; Second real-time location information determination module 63: used to determine the second real-time location information of the target storage basket based on the target category information; Sorting instruction information generation module 64: used to generate sorting instruction information based on the first real-time position information and the second real-time position information, wherein the sorting instruction information is used to instruct the target robot to grab the object to be detected and move the object to be detected from the first real-time position information to the second real-time position information.

[0069] Optionally, the system 60 also includes: Interval distance information acquisition module: used to acquire the interval distance information of the object to be detected based on a preset laser rangefinder and contact displacement sensor; The third real-time location information generation module is used to generate the third real-time location information of the object to be detected based on the interval distance information. First real-time location information determination module: used to determine whether the first real-time location information is the same as the third real-time location information; Spacing average information generation module: used to generate spacing average information based on the first real-time location information and the third real-time location information if the first real-time location information is not the same as the third real-time location information; Fourth Real-Time Location Information Generation Module: Used to generate the fourth real-time location information of the object to be detected based on the average spacing information; Fourth Real-Time Location Information Modification Module: Used to modify the first real-time location information to the fourth real-time location information.

[0070] Optionally, the system 60 also includes: Test image information acquisition module: In response to the self-test command, it continuously acquires multiple test image information based on a depth camera, wherein the shooting angles corresponding to the multiple test image information are different; Detection contour information generation module: used to generate multiple detection contour information based on a preset edge extraction algorithm for each test image; The detection contour information determination module is used to determine whether there is at least one detection contour information with the same location and size among multiple test image information; Oil stain warning information generation module: If there is at least one detectable contour information with the same location and size among multiple test image information, an oil stain warning information will be generated.

[0071] Optionally, the system 60 also includes: Oil stain removal information determination module: used to determine whether oil stain removal information has been received; Real-time depth image information acquisition module: If oil stain removal information is received, it continues to acquire real-time depth image information based on a preset depth camera; Oil stain contour removal instruction generation module: If no oil stain removal information is received, it generates an oil stain contour removal instruction based on the oil stain prompt information. The oil stain contour removal instruction is used to instruct the removal of oil stain contours in the real-time depth image information. The oil stain contours are the same contour information to be detected in multiple test image information with the same position and size.

[0072] Optionally, the system 60 also includes: Total Quantity Information Acquisition Module: Used to acquire the total quantity information and sorted quantity information corresponding to each target category; Sorting Ratio Information Generation Module: Used to generate sorting ratio information based on total quantity information and sorted quantity information; Sorting task set information generation module: used to generate sorting task set information based on the sorting ratio information corresponding to each target category; Sorting task set information sending module: used to send sorting task set information to a designated terminal.

[0073] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0074] This application also provides a terminal device, such as... Figure 7 As shown, the terminal device 70 in this embodiment includes a processor 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the processor 71. When the processor 71 executes the computer program 73, it implements the steps in the above-described intelligent control method embodiment for the sorting device, for example... Figure 1 Steps S100 to S400 are shown; or, when processor 71 executes computer program 73, it implements the functions of each module in the above-described device, for example... Figure 6 The functions of modules 61 to 64 are shown.

[0075] The terminal device 70 can be a desktop computer, laptop, handheld computer, or cloud server, etc., and includes, but is not limited to, a processor 71 and a memory 72. Those skilled in the art will understand that... Figure 7 This is merely an example of terminal device 70 and does not constitute a limitation on terminal device 70. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 70 may also include input / output devices, network access devices, buses, etc.

[0076] The processor 71 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0077] The memory 72 can be an internal storage unit of the terminal device 70, such as the hard disk or memory of the terminal device 70. The memory 72 can also be an external storage device of the terminal device 70, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 70. Furthermore, the memory 72 can include both internal storage units and external storage devices of the terminal device 70. The memory 72 can also store computer program 73 and other programs and data required by the terminal device 70. The memory 72 can also be used to temporarily store data that has been output or will be output.

[0078] One embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0079] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the methods, principles and structures of this application should be covered within the scope of protection of this application.

Claims

1. An intelligent control method for a sorting device based on a depth camera and a three-axis motion mechanism, characterized in that, The method includes: Based on a preset depth camera, acquire real-time depth image information; Based on the real-time depth image information and the preset target detection algorithm, target category information and first real-time position information of the object to be detected are generated. Based on the target category information, determine the second real-time location information of the target storage basket; Based on the first real-time location information and the second real-time location information, sorting instruction information is generated, wherein the sorting instruction information is used to instruct the target robot to grasp the object to be detected and move the object to be detected from the first real-time location information to the second real-time location information.

2. The method according to claim 1, characterized in that, After generating target category information and first real-time location information of the object to be detected based on the real-time depth image information and a preset target detection algorithm, the method further includes: Based on a preset laser rangefinder and contact displacement sensor, the interval distance information of the object to be detected is obtained; Based on the interval distance information, the third real-time position information of the object to be detected is generated; Determine whether the first real-time location information is the same as the third real-time location information; If the first real-time location information is not the same as the third real-time location information, then the average distance information is generated based on whether the first real-time location information is the same as the third real-time location information. Based on the average spacing information, the fourth real-time position information of the object to be detected is generated; The first real-time location information is modified to the fourth real-time location information.

3. The method according to claim 1, characterized in that, Before acquiring real-time depth image information based on a preset depth camera, the process includes: In response to the self-test command, multiple test image information is continuously acquired based on the depth camera, wherein the shooting angles corresponding to the multiple test image information are all different; For each of the aforementioned test image information: based on a preset edge extraction algorithm, multiple contour information to be detected is generated; Determine whether there exists at least one detectable contour information with the same location and size among the multiple test image information; If at least one detectable contour information with the same location and size exists among multiple test image information, an oil stain warning message is generated.

4. The method according to claim 3, characterized in that, After generating the oil stain warning information if at least one detectable contour information with the same location and size exists among the plurality of test image information, the method further includes: Determine whether an oil stain removal notification has been received; If the oil stain removal information is received, the process of acquiring real-time depth image information based on the preset depth camera continues; If the oil stain removal information is not received, an oil stain contour removal instruction is generated based on the oil stain prompt information. The oil stain contour removal instruction is used to instruct the removal of oil stain contours in the real-time depth image information. The oil stain contours are the same-positioned and same-sized contour information to be detected in multiple test image information.

5. The method according to claim 1, characterized in that, After generating sorting instruction information based on the first real-time location information and the second real-time location information, the method further includes: Obtain the total quantity information and sorted quantity information corresponding to each of the target categories; Based on the total quantity information and the sorted quantity information, the sorted ratio information is generated; Based on the sorting ratio information corresponding to each of the target categories, a sorting task set information is generated. Send the sorting task set information to the designated terminal.

6. An intelligent control system for a sorting device based on a depth camera and a three-axis motion mechanism, characterized in that, The system includes: Real-time depth image information acquisition module: used to acquire real-time depth image information based on a preset depth camera; Target category information generation module: used to generate target category information and first real-time position information of the object to be detected based on the real-time depth image information and the preset target detection algorithm; The second real-time location information determination module is used to determine the second real-time location information of the target storage basket based on the target category information. Sorting instruction information generation module: used to generate sorting instruction information based on the first real-time location information and the second real-time location information, wherein the sorting instruction information is used to instruct the target robot to grab the object to be detected and move the object to be detected from the first real-time location information to the second real-time location information.

7. The system according to claim 6, characterized in that, The system also includes: Total Quantity Information Acquisition Module: Used to acquire the total quantity information and sorted quantity information corresponding to each of the target category information; Sorting ratio information generation module: used to generate sorting ratio information based on the total quantity information and the sorted quantity information; Sorting task set information generation module: used to generate sorting task set information based on the sorting ratio information corresponding to each of the target category information; Sorting task set information sending module: used to send the sorting task set information to the designated terminal.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images