Bulk and general cargo identification method, device and equipment based on binocular vision and storage medium

Abstract

The invention provides a bulk cargo identification method, device and equipment based on binocular vision, and a storage medium. The method comprises the following steps: acquiring a binocular vision image of a to-be-identified area acquired based on a binocular camera; determining whether a lens of the binocular camera needs to be cleaned based on the binocular vision image; when cleaning is needed, a cleaning system is controlled to clean a lens, a cleaning binocular vision image of the to-be-recognized area is collected based on the cleaned binocular camera, and bulk goods and poses of the bulk goods in the to-be-recognized area are determined based on the cleaning binocular vision image; and when cleaning is not needed, bulk cargos and poses of the bulk cargos in the to-be-recognized area are determined based on the binocular vision image. According to the method, the definition of the visual image data for identifying the bulk goods and the poses of the bulk goods meets the requirement, the influence of dust, dirt and the like is avoided, the identification accuracy of the bulk goods and the poses of the bulk goods is improved, and the identification accuracy of the poses of the bulk goods can be further improved based on the binocular camera.

Description

Technical Field

[0001] The present invention relates to the technical field of cargo identification, and in particular to a binocular vision-based bulk cargo identification method, device, equipment and storage medium. Background Art

[0002] In bulk cargo identification scenarios such as ports and docks, accurate identification of bulk cargo is a prerequisite for grabbing the bulk cargo. The prior art proposes to identify bulk cargo through visual sensing methods, that is, to identify bulk cargo through images taken by a camera.

[0003] Due to the particularity of the working environment of ports and docks, the camera lens is easily affected by dust, dirt, etc., which causes the field of view to be blocked, resulting in inaccurate recognition of bulk cargo.

[0004] Therefore, there is an urgent need to provide a bulk cargo identification method, device, equipment and storage medium based on binocular vision, which can be used to improve the accurate identification of bulk cargo in non-dust-free environments such as ports and docks. Summary of the invention

[0005] In view of this, it is necessary to provide a method, device, equipment and storage medium for bulk cargo recognition based on binocular vision to solve the technical problem in the prior art that the influence of dust and dirt on the acquired images in non-dust-free scenes cannot be avoided, resulting in inaccurate position recognition of bulk cargo.

[0006] In order to solve the above technical problems, the present invention provides a bulk cargo recognition method based on binocular vision, comprising: Acquire a binocular visual image of the area to be identified based on the binocular camera; Determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image; When cleaning is required, controlling the cleaning system to clean the lens, collecting a clean binocular visual image of the area to be identified based on the cleaned binocular camera, and determining the bulk goods in the area to be identified and the position and posture of the bulk goods based on the clean binocular visual image; When cleaning is not required, the bulk goods in the to-be-identified area and the position and posture of the bulk goods are determined based on the binocular vision image.

[0007] In a possible implementation, the binocular vision image includes an RGB image, and determining whether the lens of the binocular camera needs to be cleaned based on the binocular vision image includes: Determine the image edge of the RGB image based on an edge detection algorithm; Extract edge features of the image edge, and determine whether the lens of the binocular camera needs to be cleaned based on the edge features.

[0008] In a possible implementation, the edge features include edge density and edge grayscale change rate; then extracting the edge features of the image includes: Acquire the number of image pixels in the RGB image and the number of edge pixels at the edge of the image, and use the ratio of the number of edge pixels to the number of image pixels as the edge density; An edge region in the RGB image is acquired, and a pixel grayscale value of each pixel in the edge region is determined, and the edge grayscale change rate is determined based on the pixel grayscale value.

[0009] In a possible implementation, determining whether the lens of the binocular camera needs to be cleaned based on the edge feature includes: Taking a weighted sum of the edge density and the edge grayscale change rate to obtain a feature comprehensive value; When the feature comprehensive value is less than a threshold value, the lens of the binocular camera needs to be cleaned; when the feature comprehensive value is greater than or equal to the threshold value, the lens of the binocular camera does not need to be cleaned.

[0010] In a possible implementation, the binocular camera includes a left camera and a right camera, the binocular visual image includes a left image corresponding to the left camera and a right image corresponding to the right camera, and determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image includes: determining whether the left camera needs to be cleaned based on the left image, and determining whether the right camera needs to be cleaned based on the right image; When the left camera needs to be cleaned, or the right camera needs to be cleaned, the lens of the binocular camera needs to be cleaned.

[0011] In a possible implementation, the binocular vision image includes an RGB image and a depth image, and the posture includes a posture and a three-dimensional position; then determining the bulk goods in the to-be-identified area and the posture of the bulk goods based on the binocular vision image includes: Inputting the RGB image into a well-trained object recognition model to obtain the bulk cargo; Acquire a bounding box of the bulk cargo, and determine target depth data in the depth image based on the bounding box; Determine the three-dimensional feature points in the target depth data and the two-dimensional feature points in the RGB image, and match the three-dimensional feature points with the two-dimensional feature points based on a feature point matching algorithm to obtain the posture of the bulk cargo; The three-dimensional feature points are converted into a camera coordinate system to obtain three-dimensional conversion feature points, and the three-dimensional position of the bulk cargo is determined based on the three-dimensional conversion feature points.

[0012] In a possible implementation, after acquiring the binocular visual image of the area to be identified acquired by the binocular camera, the method further includes: Based on the filtering algorithm and the contrast enhancement algorithm, the binocular vision image is subjected to denoising and contrast enhancement processing in sequence.

[0013] On the other hand, the present invention also provides a bulk cargo identification device based on binocular vision, comprising: A binocular vision image acquisition unit, used to acquire a binocular vision image of the area to be identified based on the binocular camera; A cleaning judgment unit, used for determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image; a first target and posture recognition unit, configured to control a cleaning system to clean the lens when cleaning is required, collect a clean binocular visual image of the area to be recognized based on the cleaned binocular camera, and determine the bulk goods and the posture of the bulk goods in the area to be recognized based on the clean binocular visual image; The second target and posture recognition unit is used to determine the bulk goods and the posture of the bulk goods in the area to be recognized based on the binocular vision image when cleaning is not required.

[0014] On the other hand, the present invention also provides a bulk cargo identification device, comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory, and is used to execute the program stored in the memory to implement the steps of the binocular vision-based bulk cargo identification method described in any one of the possible implementations above.

[0015] On the other hand, the present invention further provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the bulk cargo identification method based on binocular vision described in any one of the possible implementations described above are implemented.

[0016] The beneficial effect of adopting the above implementation mode is as follows: the binocular vision-based bulk cargo recognition method provided by the present invention, before recognizing the bulk cargo and the posture of the bulk cargo in the recognition area, first determines whether the lens of the binocular camera needs to be cleaned, so as to ensure that the clarity of the visual image data used for recognizing the bulk cargo and the posture of the bulk cargo meets the requirements, avoid the influence of dust, dirt, etc., and improve the recognition accuracy of the bulk cargo and the posture of the bulk cargo.

[0017] Furthermore, the present invention identifies bulk cargo and its posture based on binocular vision images collected by a binocular camera. Compared with a monocular camera, it can restore the three-dimensional information of the object through a stereo vision algorithm, thereby more accurately determining the position, orientation and size of the cargo, thereby effectively improving the recognition accuracy of the bulk cargo posture. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic flow chart of an embodiment of a bulk cargo identification method based on binocular vision provided by the present invention; Figure 2 A schematic diagram of the structure of an embodiment of the cleaning system provided by the present invention; Figure 3 For the present invention Figure 1 A schematic flow chart of an embodiment of step S102; Figure 4 For the present invention Figure 3 A flowchart of an embodiment of determining whether the lens of the binocular camera needs to be cleaned in step S302; Figure 5 For the present invention Figure 1 Another embodiment of step S102 Figure 6 For the present invention Figure 1 A schematic flow chart of an embodiment of step S104; Figure 7 A schematic diagram of the structure of an embodiment of a bulk cargo identification device based on binocular vision provided by the present invention; Figure 8 A schematic structural diagram of an embodiment of a bulk cargo identification device provided by the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] It should be understood that the schematic drawings are not drawn to scale. The flowchart used in the present invention shows the operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present invention, can add one or more other operations to the flowchart, and can also remove one or more operations from the flowchart. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0022] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] The present invention provides a binocular vision-based bulk cargo identification method, device, equipment and storage medium, which are described below respectively.

[0024] Figure 1 A schematic flow chart of an embodiment of the bulk cargo identification method based on binocular vision provided by the present invention is shown in FIG. Figure 1 As shown in FIG. 1 , the bulk cargo recognition method based on binocular vision includes: S101, obtaining a binocular visual image of the area to be identified acquired by a binocular camera; S102, determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image; S103, when cleaning is required, controlling the cleaning system to clean the lens, collecting a clean binocular visual image of the area to be identified based on the cleaned binocular camera, and determining the bulk goods and the position and posture of the bulk goods in the area to be identified based on the clean binocular visual image; S104: When cleaning is not required, determining the bulk goods and the positions and postures of the bulk goods in the area to be identified based on the binocular vision image.

[0025] The area to be identified in step S101 may be part or all of a range in actual application scenarios such as ports and docks, which is determined by the field of view of the binocular camera.

[0026] In a specific embodiment of the present invention, Figure 2As shown, the cleaning system includes a cleaning actuator and a cleaning drive mechanism, wherein the cleaning drive mechanism is used to generate a drive instruction in response to a cleaning instruction when the lens needs to be cleaned, and control the cleaning actuator to clean the lens. Specifically, the cleaning drive mechanism is a motor, and the cleaning actuator is a cleaning brush.

[0027] Furthermore, in order to realize the integration and installation of the cleaning actuator and the cleaning drive mechanism, the cleaning system also includes a base, and the cleaning actuator and the cleaning drive mechanism are both installed on the base.

[0028] Compared with the prior art, the binocular vision-based bulk cargo recognition method provided by the embodiment of the present invention determines whether the lens of the binocular camera needs to be cleaned before recognizing the bulk cargo and the posture of the bulk cargo in the recognition area, so as to ensure that the clarity of the visual image data used for recognizing the bulk cargo and the posture of the bulk cargo meets the requirements, avoid the influence of dust, dirt, etc., and improve the recognition accuracy of the bulk cargo and the posture of the bulk cargo.

[0029] Furthermore, the embodiments of the present invention recognize bulk cargo and its posture based on binocular vision images collected by a binocular camera. Compared with a monocular camera, the stereo vision algorithm can restore the three-dimensional information of the object, thereby more accurately determining the position, orientation and size of the cargo, thereby effectively improving the recognition accuracy of the bulk cargo posture.

[0030] Since the clarity of an image is closely related to the edge of the image, a clear image usually has a clearer and more obvious edge. Therefore, in some embodiments of the present invention, the binocular vision image includes an RGB image. Figure 3 As shown, step S102 includes: S301, determining the image edge of the RGB image based on an edge detection algorithm; S302: extract edge features of the image edge, and determine whether the lens of the binocular camera needs to be cleaned based on the edge features.

[0031] In a specific embodiment of the present invention, the edge detection algorithm is a Canny edge detection algorithm.

[0032] In order to further improve the representativeness of the extracted edge features and improve the accuracy of judging whether the lens needs to be cleaned, in some embodiments of the present invention, the edge features include edge density and edge grayscale change rate; then the edge features of the image are extracted, including: Obtain the number of image pixels in the RGB image and the number of edge pixels at the edge of the image, and take the ratio of the number of edge pixels to the number of image pixels as the edge density; The edge region in the RGB image is obtained, and the pixel grayscale value of each pixel in the edge region is determined, and the edge grayscale change rate is determined based on the pixel grayscale value.

[0033] The embodiment of the present invention uses features based on two dimensions, edge density and edge grayscale change rate, as edge features, which can improve the accuracy of edge feature expression of clarity, and further improve the accuracy of judging whether the lens needs to be cleaned.

[0034] In some embodiments of the present invention, Figure 4 As shown, determining whether the lens of the binocular camera needs to be cleaned based on the edge feature in step S302 includes: S401, performing weighted summation of edge density and edge grayscale change rate to obtain a feature comprehensive value; S402: When the feature comprehensive value is less than a threshold value, the lens of the binocular camera needs to be cleaned; when the feature comprehensive value is greater than or equal to the threshold value, the lens of the binocular camera does not need to be cleaned.

[0035] Since the higher the clarity of the RGB image, the greater the edge density and the edge grayscale change rate, the embodiment of the present invention can improve the accuracy of judging whether the lens needs to be cleaned by setting that when the feature comprehensive value is less than a threshold, it is determined that the lens needs to be cleaned.

[0036] It should be understood that the threshold value can be set or adjusted according to the actual application scenario and is not specifically limited here.

[0037] Among them, when performing weighted sum calculation on edge density and edge grayscale change rate, the weight can be set based on the actual scene and is not specifically limited here.

[0038] Since the binocular camera includes a left camera and a right camera, the binocular visual image includes a left image corresponding to the left camera and a right image corresponding to the right camera. The recognition principle of the binocular camera is: determine the depth image based on the left image and the right image, and determine the bulk goods and their positions and postures based on the depth image, the left image and the right image. Therefore, both the left image and the right image have a significant impact on the accuracy of the recognition result. In order to further ensure the recognition accuracy of the bulk goods and their positions and postures, in some embodiments of the present invention, such as Figure 5 As shown, step S102 is specifically as follows: S501, determining whether the left camera needs to be cleaned based on the left image, and determining whether the right camera needs to be cleaned based on the right image; S502: When the left camera needs to be cleaned, or the right camera needs to be cleaned, the lens of the binocular camera needs to be cleaned.

[0039] The embodiment of the present invention can ensure the accuracy of the obtained left image and right image by arranging that when the left camera needs to be cleaned or the right camera needs to be cleaned, the lens of the binocular camera needs to be cleaned, thereby improving the recognition accuracy of bulk goods and their positions and postures.

[0040] It should be noted that the cleaning execution mechanism may include a left cleaning execution sub-mechanism corresponding to the left camera and a right cleaning execution sub-mechanism corresponding to the right camera. When the left camera needs to be cleaned, the left cleaning execution sub-mechanism is controlled to perform the cleaning action. When the right camera needs to be cleaned, the right cleaning execution sub-mechanism is controlled to perform the cleaning action.

[0041] It should be noted that the principle and process of determining the bulk cargo and the position and posture of the bulk cargo in the area to be identified based on the clean binocular vision image in step S103 are the same as the principle and process of determining the bulk cargo and the position and posture of the bulk cargo in the area to be identified based on the binocular vision image in step S104. The following is an example of determining the bulk cargo and the position and posture of the bulk cargo in the area to be identified based on the binocular vision image in step S104.

[0042] In some embodiments of the present invention, the binocular vision image includes an RGB image and a depth image, and the posture includes a posture and a three-dimensional position; Figure 6 As shown, the step S104 of determining the bulk goods in the to-be-identified area and the position and posture of the bulk goods based on the binocular vision image includes: S601, inputting the RGB image into a well-trained object recognition model to obtain bulk cargo; S602: Acquire a bounding box of the bulk goods, and determine target depth data in the depth image based on the bounding box; S603, determining the three-dimensional feature points in the target depth data and the two-dimensional feature points in the RGB image, and matching the three-dimensional feature points with the two-dimensional feature points based on a feature point matching algorithm to obtain the posture of the bulk cargo; S604: Convert the three-dimensional feature points into the camera coordinate system to obtain three-dimensional conversion feature points, and determine the three-dimensional position of the bulk cargo based on the three-dimensional conversion feature points.

[0043] Among them, the target recognition model in step S601 is any target detection model in the prior art, including but not limited to: R-CNN, Fast R-CNN, Faster R-CNN, YOLO, SSD and other target detection models.

[0044] It should be noted that the bulk cargo obtained in step S601 includes the boundary box of the bulk cargo and its category information, wherein the category information is used to indicate the type of the bulk cargo.

[0045] Specifically, step S602 includes: cutting out depth data of the bounding box area from the depth image based on the bounding box information, and using the depth data as target depth data.

[0046] It should be noted that: in order to further improve the recognition accuracy of the three-dimensional position and posture, after step S602, it also includes: using a median filtering algorithm to remove abnormal values ​​in the target depth data.

[0047] In a specific embodiment of the present invention, the feature point matching algorithm is a PnP algorithm, which is a mature algorithm for implementing two-dimensional and three-dimensional data matching in the prior art, and its specific principle is not described in detail here.

[0048] Among them, the coordinate system where the three-dimensional feature points are located is the image coordinate system. Before step S604, it is necessary to first determine the conversion relationship between the image coordinate system and the camera coordinate system based on the camera intrinsic parameters, extrinsic parameters, etc., and convert the three-dimensional feature points into the camera coordinate system based on the conversion relationship.

[0049] Since the image quality of the binocular vision image is closely related to the accuracy of bulk cargo and its position and posture recognition, in some embodiments of the present invention, after step S101, the following is further included: Based on the filtering algorithm and contrast enhancement algorithm, the binocular vision image is denoised and contrast enhanced in turn.

[0050] Among them, the filtering algorithm is a bilateral filtering algorithm, which can retain edge details while removing noise, facilitating subsequent clarity judgment.

[0051] Among them, the contrast enhancement algorithm is the Contrast Constrained Adaptive Histogram Equalization (CLAHE) algorithm.

[0052] The embodiment of the present invention performs denoising and contrast enhancement processing on binocular visual images, thereby eliminating part of the image noise and improving the image quality, thereby improving the accuracy of subsequent recognition of bulk goods and their position and posture.

[0053] In summary, the binocular vision-based bulk cargo recognition method provided by the embodiment of the present invention adopts a binocular vision algorithm to effectively improve the recognition accuracy of the bulk cargo position, thereby improving the accuracy and efficiency of bulk cargo grabbing. In addition, by setting a determination of whether the lens needs to be cleaned before recognition, it can automatically clean the camera lens when it is blocked by dust, maintain the clarity of the camera lens, ensure the stability, reliability and accuracy of the recognition effect, avoid frequent manual cleaning operations, and improve work efficiency and comfort. The embodiment of the present invention innovatively solves the problems of low accuracy of traditional bulk cargo recognition technology and dust affecting recognition accuracy, and provides an efficient and accurate solution for grabbing bulk cargo in dusty work places such as ports and docks.

[0054] In order to better implement the bulk cargo identification method based on binocular vision in the embodiment of the present invention, on the basis of the bulk cargo identification method based on binocular vision, the embodiment of the present invention also provides a bulk cargo identification device based on binocular vision, such as Figure 7 As shown, the binocular vision-based bulk cargo identification device 700 includes: A binocular vision image acquisition unit 701 is used to acquire a binocular vision image of the area to be identified based on the binocular camera; A cleaning judgment unit 702 is used to determine whether the lens of the binocular camera needs to be cleaned based on the binocular vision image; The first target and posture recognition unit 703 is used to control the cleaning system to clean the lens when cleaning is required, collect a clean binocular visual image of the area to be recognized based on the cleaned binocular camera, and determine the bulk goods and the posture of the bulk goods in the area to be recognized based on the clean binocular visual image; The second target and position and posture recognition unit 704 is used to determine the bulk goods and the position and posture of the bulk goods in the area to be recognized based on the binocular vision image when cleaning is not required.

[0055] The binocular vision-based bulk cargo identification device 700 provided in the above embodiment can implement the technical solution described in the above embodiment of the binocular vision-based bulk cargo identification method. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the binocular vision-based bulk cargo identification method, which will not be repeated here.

[0056] like Figure 8 As shown, the present invention also provides a bulk cargo identification device 800. The bulk cargo identification device 800 includes a processor 801, a memory 802 and a display 803. Figure 8 Only some of the components of the bulk cargo identification device 800 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0057] In some embodiments, the processor 801 may be a central processing unit (CPU), a microprocessor or other data processing chip, and is used to run program codes or process data stored in the memory 802, such as the bulk cargo recognition method based on binocular vision in the present invention.

[0058] In some embodiments, the processor 801 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 801 may be local or remote. In some embodiments, the processor 801 may be implemented in a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0059] In some embodiments, the memory 802 may be an internal storage unit of the bulk cargo identification device 800, such as a hard disk or memory of the bulk cargo identification device 800. In other embodiments, the memory 802 may also be an external storage device of the bulk cargo identification device 800, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the bulk cargo identification device 800.

[0060] Furthermore, the memory 802 may include both an internal storage unit of the bulk cargo identification device 800 and an external storage device. The memory 802 is used to store application software installed in the bulk cargo identification device 800 and various data.

[0061] In some embodiments, the display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 803 is used to display information of the bulk cargo identification device 800 and to display a visual user interface. The components 801-803 of the bulk cargo identification device 800 communicate with each other via a system bus.

[0062] In some embodiments of the present invention, when the processor 801 executes the binocular vision-based bulk cargo recognition program in the memory 802, the following steps may be implemented: Acquire a binocular visual image of the area to be identified based on the binocular camera; Determine whether the lens of the binocular camera needs to be cleaned based on the binocular vision image; When cleaning is required, the cleaning system is controlled to clean the lens, a clean binocular visual image of the area to be identified is collected based on the cleaned binocular camera, and the bulk goods and their positions and postures in the area to be identified are determined based on the clean binocular visual image; When cleaning is not required, the bulk goods and their positions and postures in the area to be identified are determined based on the binocular vision image.

[0063] It should be understood that: when the processor 801 executes the binocular vision-based bulk cargo recognition program in the memory 802, in addition to the above functions, other functions may also be implemented, and details may refer to the description of the corresponding method embodiments above.

[0064] Accordingly, an embodiment of the present invention further provides a computer-readable storage medium, which is used to store a computer-readable program or instruction. When the program or instruction is executed by a processor, the steps or functions of the bulk cargo identification method based on binocular vision provided by the above-mentioned method embodiments can be implemented.

[0065] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0066] The above is a detailed introduction to a bulk cargo identification method, device, equipment and storage medium based on binocular vision provided by the present invention. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying bulk cargo based on binocular vision, characterized in that: include: Acquire a binocular visual image of the area to be identified based on the binocular camera; Determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image; When cleaning is required, controlling the cleaning system to clean the lens, collecting a clean binocular visual image of the area to be identified based on the cleaned binocular camera, and determining the bulk goods in the area to be identified and the position and posture of the bulk goods based on the clean binocular visual image; When cleaning is not required, the bulk goods in the to-be-identified area and the position and posture of the bulk goods are determined based on the binocular vision image.

2. The binocular vision-based bulk cargo identification method according to claim 1, characterized in that: The binocular vision image includes an RGB image, and determining whether the lens of the binocular camera needs to be cleaned based on the binocular vision image includes: Determine the image edge of the RGB image based on an edge detection algorithm; Extract edge features of the image edge, and determine whether the lens of the binocular camera needs to be cleaned based on the edge features.

3. The binocular vision-based bulk cargo identification method according to claim 2, characterized in that: The edge features include edge density and edge grayscale change rate; then extracting the edge features of the image includes: Acquire the number of image pixels in the RGB image and the number of edge pixels at the edge of the image, and use the ratio of the number of edge pixels to the number of image pixels as the edge density; An edge region in the RGB image is acquired, and a pixel grayscale value of each pixel in the edge region is determined, and the edge grayscale change rate is determined based on the pixel grayscale value.

4. The binocular vision-based bulk cargo identification method according to claim 3 is characterized in that: The determining whether the lens of the binocular camera needs to be cleaned based on the edge feature includes: Taking a weighted sum of the edge density and the edge grayscale change rate to obtain a feature comprehensive value; When the feature comprehensive value is less than a threshold value, the lens of the binocular camera needs to be cleaned; when the feature comprehensive value is greater than or equal to the threshold value, the lens of the binocular camera does not need to be cleaned.

5. The binocular vision-based bulk cargo identification method according to claim 1, characterized in that: The binocular camera includes a left camera and a right camera, and the binocular visual image includes a left image corresponding to the left camera and a right image corresponding to the right camera. Then, determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image includes: determining whether the left camera needs to be cleaned based on the left image, and determining whether the right camera needs to be cleaned based on the right image; When the left camera needs to be cleaned, or the right camera needs to be cleaned, the lens of the binocular camera needs to be cleaned.

6. The binocular vision-based bulk cargo identification method according to claim 1, characterized in that: The binocular vision image includes an RGB image and a depth image, and the posture includes a posture and a three-dimensional position; then determining the bulk goods in the to-be-identified area and the posture of the bulk goods based on the binocular vision image includes: Inputting the RGB image into a well-trained object recognition model to obtain the bulk cargo; Acquire a bounding box of the bulk cargo, and determine target depth data in the depth image based on the bounding box; Determine the three-dimensional feature points in the target depth data and the two-dimensional feature points in the RGB image, and match the three-dimensional feature points with the two-dimensional feature points based on a feature point matching algorithm to obtain the posture of the bulk cargo; The three-dimensional feature points are converted into a camera coordinate system to obtain three-dimensional conversion feature points, and the three-dimensional position of the bulk cargo is determined based on the three-dimensional conversion feature points.

7. The binocular vision-based bulk cargo identification method according to claim 1, characterized in that: After acquiring the binocular visual image of the area to be identified collected by the binocular camera, the method further includes: Based on the filtering algorithm and the contrast enhancement algorithm, the binocular vision image is subjected to denoising and contrast enhancement processing in sequence.

8. A bulk cargo identification device based on binocular vision, characterized in that: include: A binocular vision image acquisition unit, used to acquire a binocular vision image of the area to be identified based on the binocular camera; A cleaning judgment unit, used for determining whether the lens of the binocular camera needs to be cleaned based on the binocular visual image; a first target and posture recognition unit, configured to control a cleaning system to clean the lens when cleaning is required, collect a clean binocular visual image of the area to be recognized based on the cleaned binocular camera, and determine the bulk goods and the posture of the bulk goods in the area to be recognized based on the clean binocular visual image; The second target and posture recognition unit is used to determine the bulk goods and the posture of the bulk goods in the area to be recognized based on the binocular vision image when cleaning is not required.

9. A bulk cargo identification device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the binocular vision-based bulk cargo identification method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the bulk cargo identification method based on binocular vision described in any one of claims 1 to 7 are implemented.

Images