An automated appearance inspection method and apparatus

Abstract

The application relates to the technical field of automatic sorting, and provides an automatic appearance detection method and device, the method comprising the following steps: acquiring image data of a logistics packaging box by using an image acquisition device; performing image recognition on the image data by using an object recognition model to obtain objects contained in the logistics packaging box; extracting the objects from the image data for analysis to obtain size information of the objects; detecting the filling states of respective sorting containers by using sensors; and determining a sorting strategy of the objects by using a preset algorithm according to the size information of the objects and the filling states of the respective sorting containers. The size information of the objects and the filling states of the sorting containers are analyzed, so that the sorting strategy can be determined, and the possibility of container burst in large-scale sorting can be avoided.

Description

Technical Field

[0001] This invention relates to the field of automated sorting technology, and in particular to an automated appearance inspection method and apparatus. Background Technology

[0002] Intelligent sorting refers to the automatic completion of all operations from the moment goods enter the sorting equipment until they reach the designated location, according to instructions. Therefore, this sorting method has strong processing capacity, can sort a wide range of goods in large quantities, and is currently the preferred sorting method for logistics sorting centers.

[0003] Intelligent sorting requires the use of automated appearance inspection robots, which play a crucial role in the process. These robots utilize advanced vision technology and image processing algorithms to quickly and accurately detect the appearance features of goods, such as shape, color, and labels. Connected to the sorting system, the appearance inspection robots can acquire real-time information about the goods' appearance and compare it with preset standards. If the appearance of a product does not conform to the standards, the robot automatically sorts it into the appropriate anomaly handling area for further processing.

[0004] However, traditional rule engines and traditional computer vision methods often struggle to handle complex scenes and changing appearances, resulting in limited accuracy in object recognition. Furthermore, traditional sorting strategies are often based on simple rules or experience, lacking intelligence and adaptability, which can lead to overload of sorting containers used to store boxes of corresponding sizes during large-scale sorting processes.

[0005] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0006] The technical problem that this invention aims to solve is that existing rule engines and traditional computer vision methods often struggle to handle complex scenes and changing appearances, resulting in limited accuracy in object recognition. Furthermore, existing sorting strategies are often based on simple rules or experience, lacking intelligence and adaptability, which can lead to the possibility of overflowing sorting containers used to store boxes of corresponding sizes during large-scale sorting processes.

[0007] The present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides an automated appearance inspection method, comprising:

[0009] Use image acquisition equipment to acquire image data of logistics packaging boxes;

[0010] The image data is used to perform image recognition using an object recognition model to obtain the objects contained in the logistics packaging box;

[0011] The object is extracted from the image data and analyzed to obtain the object's size information;

[0012] Sensors are used to detect the filling status of each sorting container;

[0013] Based on the size information of the object and the filling status of each sorting container, a preset algorithm is used to determine the sorting strategy for the object.

[0014] Preferably, multiple image acquisition devices are used to acquire image data of the logistics packaging box from multiple different acquisition angles to obtain the objects contained in the multiple image data.

[0015] The step of extracting and analyzing the object from the image data to obtain the object's size information specifically includes: calculating the consistency of the objects contained in each image data; determining whether the objects contained in multiple image data are the same object based on the consistency; if the objects are the same object, matching the same object contained in multiple image data to obtain a three-dimensional model of the object; and analyzing the three-dimensional model to obtain the object's size information.

[0016] Preferably, the object recognition model is a deep learning model;

[0017] The deep learning model is pre-trained using an image dataset to facilitate image recognition using the trained deep learning model.

[0018] Preferably, a laser rangefinder is also installed at the location of the image acquisition device;

[0019] The laser rangefinder is used to measure the distance between the image acquisition device and the object in the logistics packaging box, so that the object's size information can be obtained by jointly analyzing the distance information and image data.

[0020] Preferably, the step of using sensors to detect the filling status of each sorting container specifically involves:

[0021] The weight of the sorting container is detected using a weight sensor, and the weight information is compared with the maximum load capacity of the sorting container to determine the filling status of the sorting container; or,

[0022] The size of the space occupied in the sorting container is scanned using a laser scanner, and the size of the occupied space is compared with the size of the sorting container itself to obtain the filling status of the sorting container.

[0023] Preferably, the step of determining the sorting strategy for the object using a preset algorithm based on the object's size information and the filling status of each sorting container specifically includes:

[0024] Based on the category of the object, identify each first sorting container for containing the object of that category, and determine whether there is a first sorting container capable of containing the object based on the filling status of each first sorting container and the size information of the object.

[0025] If a first sorting container exists that can hold the object, sort the object into the first sorting container that can hold the object;

[0026] If there is no first sorting container that can accommodate the object, the sorting strategy for the object is determined based on the object's own safety factor, the object's influence factor on the outside world, the safety factor of the objects already in each second sorting container, and the influence factor of the objects already in each second sorting container on the outside world; wherein, the second sorting container is a sorting container used to accommodate other types of objects.

[0027] Preferably, the sorting strategy for the object is determined based on the object's own safety factor, the object's influence factor on the external environment, the safety factor of existing objects in each second sorting container, and the influence factor of existing objects in each second sorting container on the external environment. Specifically, this includes:

[0028] The safety factor of the second sorting container is the one with the highest safety factor among the existing objects in the second sorting container.

[0029] The influence coefficient of the second sorting container is the one with the lowest influence coefficient on the outside world.

[0030] Using the condition that the influence coefficient of the second sorting container is not lower than the safety factor of the object itself, and the safety factor of the second sorting container is not higher than the influence coefficient of the object on the outside world, the second sorting containers are screened to obtain the third sorting container that meets the screening conditions.

[0031] Based on the filling status of each third sorting container and the size information of the object, determine whether there is a third sorting container that can accommodate the object.

[0032] If a third sorting container exists that can hold the object, then the object is sorted into the third sorting container that can hold the object.

[0033] Preferably, when there are multiple first sorting containers capable of holding the object, sorting the object into the first sorting container specifically includes:

[0034] Calculate the capacity coefficient of each first sorting container that can hold the object, and sort the object into the first sorting container with the highest capacity coefficient;

[0035] Wherein, the tolerance coefficient Where k1 and k2 are preset coefficients, V o V represents the object's size information. c The first sorting container is filled, and Dis(o,c) is the distance between the object and the first sorting container.

[0036] Secondly, the present invention provides an automated appearance inspection device, including an image acquisition module, an object recognition module, a size measurement module, a sorting container management module, and a control and decision-making module;

[0037] The image acquisition module is used to acquire image data of logistics packaging boxes using image acquisition equipment;

[0038] The size measurement module is used to perform image recognition on the image data using an object recognition model to obtain the objects contained in the logistics packaging box; and to extract and analyze the objects from the image data to obtain the size information of the objects.

[0039] The sorting container management module is used to detect the filling status of each sorting container using sensors;

[0040] The control and decision module is used to determine the sorting strategy for the object based on the object's size information and the filling status of each sorting container using a preset algorithm.

[0041] Thirdly, the present invention also provides an automated appearance inspection device for implementing the automated appearance inspection method described in the first aspect, the device comprising:

[0042] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor for performing the automated appearance inspection method described in the first aspect.

[0043] Fourthly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the automated appearance inspection method described in the first aspect.

[0044] This invention analyzes the size information of objects and the filling status of sorting containers to determine sorting strategies and avoid the possibility of sorting containers overflowing during large-scale sorting. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0046] Figure 1 This is a flowchart illustrating an automated appearance inspection method provided in an embodiment of the present invention;

[0047] Figure 2 This is a flowchart illustrating an automated appearance inspection method provided in an embodiment of the present invention;

[0048] Figure 3 This is a flowchart illustrating an automated appearance inspection method provided in an embodiment of the present invention;

[0049] Figure 4 This is a flowchart illustrating an automated appearance inspection method provided in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of an automated appearance inspection method provided in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the architecture of an automated appearance inspection device provided in an embodiment of the present invention;

[0052] Figure 7 This is a flowchart illustrating an automated appearance inspection method provided in an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram of the architecture of an automated appearance inspection device provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0055] In this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0056] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] Example 1:

[0058] Existing rule engines and traditional computer vision methods often struggle to handle complex scenes and changing appearances, resulting in limited accuracy in object recognition. Furthermore, existing sorting strategies are often based on simple rules or experience, lacking intelligence and adaptability. This can lead to overflowing sorting containers used to store boxes of corresponding sizes during large-scale sorting processes. To address this issue, Embodiment 1 of this invention provides an automated appearance inspection method, such as... Figure 1 As shown, it includes:

[0059] In step 201, image data of the logistics packaging box is acquired using an image acquisition device. This image acquisition device can be a camera sensor, such as a Complementary Metal-Oxide-Semiconductor (CMOS) sensor or a Charge-Coupled Device (CCD) sensor. The camera acquires high-resolution image data of the logistics packaging box through appropriate lens settings and light source adjustment. In a preferred embodiment, suitable image processing techniques, such as noise reduction and contrast enhancement, can also be used to improve image quality. Acquiring the image data of the logistics packaging box actually involves acquiring images of the objects contained within the box.

[0060] In step 202, an object recognition model is used to perform image recognition on the image data to obtain the objects contained in the logistics packaging box; the object recognition model can be a deep learning model, such as a convolutional neural network, and the actual result of image recognition is the category label and confidence level of a certain region in the image belonging to a certain object.

[0061] In step 203, the object is extracted from the image data and analyzed to obtain the object's size information.

[0062] In step 204, sensors are used to detect the filling status of each sorting container. This detection can be achieved by: using a weight sensor to detect the weight information of the sorting container and comparing the weight information with the maximum load capacity of the sorting container to obtain the filling status; or, using a laser scanner to scan the occupied space within the sorting container and comparing the occupied space with the container's own size to obtain the filling status. The maximum load capacity and the container's own size are determined by those skilled in the art based on the characteristics of the sorting container.

[0063] In step 205, based on the size information of the object and the filling status of each sorting container, a preset algorithm is used to determine the sorting strategy for the object. The preset algorithm is derived by those skilled in the art based on experience. One possible implementation is to sort the object into a sorting container whose filling status is sufficient to accommodate the object.

[0064] The method described in this embodiment is applicable to automated appearance inspection robots. By analyzing the size information of objects and the filling status of sorting containers, a sorting strategy can be determined, thus avoiding the possibility of sorting containers overflowing during large-scale sorting.

[0065] In a preferred embodiment, multiple image acquisition devices can be used to acquire image data of the logistics packaging box from multiple different acquisition angles, obtaining objects contained in multiple image data sets; the objects are then extracted from the image data and analyzed to obtain the object's size information, such as... Figure 2 As shown, it specifically includes:

[0066] In step 301, the consistency of objects contained in each image data is calculated, and the consistency is used to determine whether the objects contained in multiple image data are the same object. The consistency can be obtained through the analysis of multiple features of the object, such as the outer contour of the object, feature points, etc.

[0067] In step 302, if the images are determined to be the same object, the images containing the same object are matched to obtain a 3D model of the object. The 3D model is then analyzed to obtain the object's size information. The 3D model is obtained by analyzing and synthesizing the contour or shape information of the same object contained in multiple image data, with the aim of restoring the object's 3D shape and spatial dimensions.

[0068] In an optional implementation, the object recognition model is a deep learning model; the deep learning model is pre-trained using an image dataset so that the trained deep learning model can be used for image recognition.

[0069] In a preferred embodiment, a laser rangefinder is also installed at the location of the image acquisition device. The laser rangefinder is used to measure the distance between the image acquisition device and the object in the logistics packaging box, so that the object's size information can be obtained by jointly analyzing the distance information and image data. The process of using the object's distance information and image data to obtain the object's size information can involve incorporating the distance information between the object and the image acquisition device into the synthesis of the 3D model in step 302 above, thereby making the generated 3D model more accurate and more realistically reproducing the object's form.

[0070] Considering that in practical use, during large-scale sorting, there are often multiple types of objects to be sorted, and for ease of management, similar objects are often placed in the same sorting container, this embodiment provides the following preferred implementation method: based on the size information of the object and the filling state of each sorting container, a preset algorithm is used to determine the sorting strategy for the object, such as... Figure 3 As shown, it specifically includes:

[0071] In step 401, each first sorting container for containing objects of that category is identified according to the category of the object. Based on the filling status of each first sorting container and the size information of the object, it is determined whether there is a first sorting container capable of containing the object. The sorting container can be pre-defined by those skilled in the art to contain which category of object.

[0072] In step 402, if there is a first sorting container capable of holding the object, the object is sorted into the first sorting container capable of holding the object.

[0073] In step 403, if there is no first sorting container that can accommodate the object, the sorting strategy for the object is determined based on the object's own safety factor, the object's influence factor on the outside world, the safety factor of the objects already in each second sorting container, and the influence factor of the objects already in each second sorting container on the outside world; wherein, the second sorting container is a sorting container used to accommodate other types of objects.

[0074] The safety factor of an object can be understood as the potential loss that the object may suffer when it is damaged. The higher the loss that the object may suffer when it is damaged, the higher the safety factor of the object. Conversely, the lower the loss that the object may suffer when it is damaged, the higher the safety factor of the object. The loss mentioned is often an economic loss. For example, the higher the price of the object, the higher its safety factor.

[0075] The influence coefficient of an object on the external environment represents the degree of impact it will have on the outside world when the object is damaged due to external conditions or other reasons, as well as the probability of the object being damaged. For example, if an object containing liquid is damaged, the liquid will leak out, affecting other objects in the same sorting container. To facilitate comparison with the object's own safety factor, the more likely the object is to be damaged, and the greater the impact it will have on the outside world when damaged, the smaller the influence coefficient. Conversely, the less likely the object is to be damaged, and the less its impact on the outside world, the larger the influence coefficient. In this case, for ease of understanding, the influence coefficient can be interpreted as the object's safety level relative to the outside world. The corresponding object's own safety factor and influence coefficient are obtained by those skilled in the art based on the object's category.

[0076] This embodiment prioritizes storing objects in sorting containers designed to hold that type of object. If an object cannot be sorted into a sorting container of its own type, a sorting strategy is determined by comprehensively considering the object's own safety, its impact on the outside world, the safety of the object in the sorting container, and its impact on the outside world. This ensures the safety of each sorted object and reduces losses caused by damage, thereby maximizing object sorting while ensuring object safety.

[0077] In practical use, the sorting strategy for the object is jointly determined based on the object's own safety factor, the object's influence factor on the external environment, the safety factor of existing objects in each second sorting container, and the influence factor of existing objects in each second sorting container on the external environment. Figure 4 As shown, it specifically includes:

[0078] In step 501, the safety factor of the second sorting container is the one with the highest safety factor among the existing objects in the second sorting container.

[0079] In step 502, the influence coefficient of the object in the second sorting container that has the lowest influence coefficient on the outside world is taken as the influence coefficient of the second sorting container.

[0080] In step 503, the second sorting container is selected based on the following criteria: the influence coefficient of the second sorting container is not lower than the object's own safety coefficient, and the safety coefficient of the second sorting container is not higher than the object's influence coefficient on the outside world. This is used to select third sorting containers that meet these criteria. Since objects with low influence coefficients are present in the second sorting container, these objects are considered to have low safety levels, are easily damaged, and cause significant impact on the outside world. If an object with a high safety coefficient is placed in the second sorting container, damage to the object with the low safety coefficient could cause significant loss to the object being sorted. Conversely, if the influence coefficient of the object being sorted is low, it is considered to have low safety levels, are easily damaged, and cause significant impact on the outside world. If an object with a high safety coefficient is present in the second sorting container, damage to the object being sorted could easily cause loss to the object with the high safety coefficient. Therefore, this embodiment uses two aspects to limit the safety of both the object being sorted and the objects already in the sorting container.

[0081] In step 504, based on the filling status of each third sorting container and the size information of the object, it is determined whether there is a third sorting container that can accommodate the object.

[0082] In step 505, if a third sorting container exists that can hold the object, the object is sorted into the third sorting container that can hold the object.

[0083] by Figure 5 As shown in the example, each small square represents an object, the lower left corner of the small square represents the object's own safety factor, and the upper right corner of the small square represents the object's influence factor on the outside world.

[0084] Figure 5 There are two sorting containers, container_1 and container_2. In container_1, the highest self-safety factor among existing objects is 3, and the lowest influence factor on the external environment is 2. Similarly, in container_2, the highest self-safety factor is 2, and the lowest influence factor is 3. In container_3, the highest self-safety factor is 3, and the lowest influence factor is 1. In container_4, the highest self-safety factor is 4, and the lowest influence factor is 3. The object to be sorted has a self-safety factor of 3 and an influence factor of 3. Only container_2 satisfies the influence factor of the second sorting container (e.g., ...). Figure 5 The value of 3) is not lower than the object's own safety factor (e.g., Figure 5 The value of the middle is 3), and the safety factor of the second sorting container itself (such as...) Figure 5 The value of 2) is not higher than the coefficient of influence of the object on the external environment (e.g., 2). Figure 5 Since the value in the middle is 3), container_2 is the third sorting container.

[0085] In practical use, when there are multiple first sorting containers capable of holding the object, the process of sorting the object into the first sorting container capable of holding the object specifically includes:

[0086] Calculate the capacity coefficient of each first sorting container capable of holding the object, and sort the object into the first sorting container with the highest capacity coefficient; wherein, the capacity coefficient... Where k1 and k2 are preset coefficients, V o V represents the object's size information. c The first sorting container is filled, and Dis(o,c) is the distance between the object and the first sorting container.

[0087] Similarly, when there are multiple third sorting containers capable of holding the object, sorting the object into the third sorting container specifically includes:

[0088] Calculate the capacity coefficient of each third sorting container capable of holding the object, and sort the object into the first sorting container with the highest capacity coefficient; wherein, the capacity coefficient... Where k1 and k2 are preset coefficients, V o V represents the object's size information. c The third sorting container is filled, and Dis(o,c) is the distance between the object and the third sorting container.

[0089] Example 2:

[0090] Based on the automated appearance inspection method described in Embodiment 1, this embodiment also provides an automated appearance inspection device for implementing the automated appearance inspection method described in Embodiment 1. The device includes an image acquisition module, an object recognition module, a size measurement module, a sorting container management module, and a control and decision-making module.

[0091] The image acquisition module is used to acquire image data of logistics packaging boxes using an image acquisition device; the size measurement module is used to perform image recognition on the image data using an object recognition model to obtain the objects contained in the logistics packaging boxes; and to extract and analyze the objects from the image data to obtain the size information of the objects; the sorting container management module is used to detect the filling status of each sorting container using sensors; the control and decision module is used to determine the sorting strategy of the objects based on the size information of the objects and the filling status of each sorting container using a preset algorithm.

[0092] In practical use, such as Figure 6 As shown, the device also includes a control and decision-making module, a human-machine interface module, and a data storage and analysis module, specifically:

[0093] The image acquisition module employs a high-resolution camera sensor, such as a CMOS or CCD sensor. The camera acquires high-resolution image data of the logistics packaging boxes through appropriate lens settings and light source adjustment. Suitable image processing techniques, such as noise reduction and contrast enhancement, are used to improve image quality. High-resolution images provide more detailed information, thereby improving the accuracy of object recognition and size measurement.

[0094] The object recognition module is used to accurately identify objects based on a trained deep learning model, such as a convolutional neural network. It uses open-source deep learning frameworks, such as TensorFlow or PyTorch, for model training and inference. During training, the model is iteratively trained using a large-scale image dataset to improve the accuracy and robustness of object recognition. The object recognition module outputs the object's class label and confidence score, and uses standard object recognition evaluation metrics, such as precision and recall, to evaluate model performance.

[0095] The size measurement module extracts the object's size information using computer vision technology based on the object recognition results. For example, it uses edge detection algorithms, such as the Canny algorithm, to find the object's boundaries and calculate dimensional parameters such as the boundary's length, width, and height. It also uses shape matching algorithms, such as the Hough transform, to fit the object's shape, resulting in more accurate size measurements. Furthermore, it combines laser rangefinders or depth sensors, such as structured light or Time-of-Flight (ToF) sensors, for non-contact size measurement to improve accuracy.

[0096] The sorting container management module monitors the filling status, weight, and size requirements of sorting containers in real time, and adjusts container positions and selects appropriate container types through intelligent algorithms. Weighing sensors measure the weight of containers, and weight thresholds are used to determine if a container is full. For size requirements, minimum and maximum size limits are set for containers, and the appropriate container type is automatically selected based on object recognition and size measurement results. Intelligent container management avoids overfilling or waste, improves logistics efficiency, and saves costs.

[0097] The control and decision-making module is used to make intelligent sorting decisions based on the results of object recognition and size measurement, using a rule engine and machine learning algorithms. The rule engine defines the system's operating rules and algorithms, such as assigning objects to appropriate sorting areas based on object category and size. Furthermore, machine learning algorithms, such as decision trees and support vector machines, are used to learn and optimize sorting decisions. Through continuous training and optimization, the accuracy and efficiency of sorting decisions are improved.

[0098] The human-machine interface (HMI) module provides an intuitive operating interface and real-time information display through touchscreen or voice interaction, facilitating operator monitoring and management of the sorting process. The interface displays the results of object recognition and size measurement, the status and location of containers, and the execution status of sorting decisions. Operators can perform parameter settings, handle anomalies, and other operations via touchscreen or voice commands, thereby achieving real-time monitoring and management of the sorting process.

[0099] The data storage and analysis module stores acquired image data, object recognition results, and dimensional measurement data in a database, and provides data analysis tools for performance evaluation, system optimization, and improvement. By storing and analyzing data, the module helps understand the system's operation and performance, identifying potential problems and areas for improvement. It also utilizes data analysis tools such as data mining and machine learning to analyze large amounts of data, uncovering hidden correlations and further optimizing system performance and algorithm accuracy.

[0100] This invention, by combining an object recognition module and a size measurement module, can accurately identify objects and measure their dimensions, avoiding misjudgments or measurement errors that occur during manual operation, thus improving the accuracy and consistency of inspection. Through an intelligent container management module and a control and decision-making module, the system can monitor the container's filling status, weight, and size requirements in real time, and automatically adjust the container position and select the appropriate container type based on the results of object recognition and size measurement. This avoids overfilling or waste, improves sorting efficiency, and reduces logistics processing time and costs. By integrating these modules, the system can automate the appearance inspection and sorting process, reducing manual operation and errors. Automated processing improves work efficiency, reduces labor costs, and provides a more reliable and efficient solution for the logistics industry.

[0101] As one embodiment of the present invention, the image acquisition module includes multiple camera sensors, which improve the accuracy of recognition and measurement by simultaneously acquiring images from multiple angles.

[0102] Specifically, the image acquisition module uses multiple camera sensors, such as two or more cameras, mounted at different angles and positions. These cameras begin image acquisition simultaneously via a synchronized triggering mechanism, and have the same exposure time and frame rate to maintain image consistency. Each camera is equipped with a suitable lens and light source to ensure image quality and detail capture of objects at different angles.

[0103] In object recognition and dimensional measurement, images from multiple angles provide more information and perspectives, thereby enhancing the accuracy of recognition and measurement. For example, the appearance and features of an object vary with the angle; by acquiring images from multiple angles and integrating information from different perspectives, the accuracy of object recognition is improved. Furthermore, for dimensional measurement, images from multiple angles provide more perspectives and boundary information, thus improving the accuracy of dimensional measurement.

[0104] The use of multiple camera sensors provides stereoscopic vision, similar to human binocular vision. By processing and analyzing images from multiple angles, depth information and three-dimensional structure of objects are obtained. This stereoscopic vision principle is used for object recognition and size measurement.

[0105] In object recognition, features such as shape, texture, and color of objects are determined by comparing images from different angles. For example, by calculating the differences and consistency between images, the shape contours of objects are detected and different object categories are distinguished. This improves the robustness and accuracy of object recognition.

[0106] In terms of dimensional measurement, a 3D model or point cloud representation of an object is established by registering and matching images from multiple angles. Then, the object's dimensions are measured by calculating the geometric properties of the model or point cloud, such as boundaries and volume. This stereoscopic vision-based dimensional measurement method provides more accurate measurement results and avoids measurement errors from a single viewpoint.

[0107] In conclusion, employing multiple camera sensors for image acquisition can improve the accuracy of object recognition and size measurement, and enhance the comprehensive understanding of object information.

[0108] As one embodiment of the present invention, the object recognition module is based on transfer learning of deep learning models, and improves the recognition ability of different objects and scenes by pre-training on large-scale datasets.

[0109] Specifically, the object recognition module uses deep learning models, such as convolutional neural networks (CNNs) or pre-trained image classification models. These models are pre-trained on large-scale image datasets, such as the ImageNet dataset, to learn general object features and visual representations.

[0110] In practical applications, the object recognition module uses transfer learning to apply pre-trained models to specific scenarios and tasks. Specifically, the weight parameters of the pre-trained model are imported into the object recognition module and fine-tuned or adjusted according to actual needs. This fine-tuning process typically involves training on a small, specific dataset to adapt to specific object categories and scenarios.

[0111] To improve the ability to recognize different objects and scenes, data augmentation techniques, such as random cropping, rotation, and scaling, are employed to generate more diverse and richer training samples. This increases the model's adaptability to changes and diversity in objects.

[0112] Deep learning models learn high-level feature representations and abstract concepts from raw image data through the combination of multiple layers of neural networks. Pre-trained models, trained on large-scale datasets, are able to learn general object features and visual representations. These features include edges, textures, shapes, etc., capturing different visual attributes of objects.

[0113] Through transfer learning, when a pre-trained model is applied to a specific object recognition task, the model utilizes the learned general features to quickly adapt to new object categories and scenes. The fine-tuning process adjusts the model's parameters to better suit the requirements of the specific task, improving the accuracy and robustness of object recognition.

[0114] In summary, transfer learning of deep learning models effectively improves the performance of object recognition modules, reduces data requirements, and adapts to the recognition requirements of different objects and scenarios, bringing significant benefits to the control system of automated appearance inspection robots.

[0115] As one embodiment of the present invention, the size measurement module is combined with a laser rangefinder or a depth sensor to achieve accurate size measurement.

[0116] Specifically, the size measurement module uses a laser rangefinder or depth sensor as the measuring device. These devices measure the distance between the object and the sensor by emitting laser beams or infrared light and utilizing the principles of light reflection or time difference.

[0117] In practical applications, the size measurement module combines a laser rangefinder or depth sensor with an image acquisition module. The image acquisition module acquires an image of the object, while the laser rangefinder or depth sensor provides distance information between the object and the sensor. By correlating the image and distance information, accurate measurement of the object's size is achieved.

[0118] During the measurement process, calibration and adjustment are used to ensure accuracy. Calibration involves measuring an object of known size to determine parameters of the laser rangefinder or depth sensor, such as field of view and distortion. Adjustment is achieved by comparing the measurement results with the actual dimensions to improve measurement precision.

[0119] Laser rangefinders or depth sensors use the speed of light to measure the distance between an object and the sensor. Laser rangefinders typically calculate distance by emitting a laser beam and measuring the time it takes for the beam to travel from emission to reception. Depth sensors, on the other hand, use technologies such as infrared light or structured light to obtain distance information by measuring the reflection time of light or the projection pattern of light.

[0120] By combining an image acquisition module, distance information is correlated with the object's image. By identifying feature points or boundaries of the object in the image and integrating this information with the distance data, the object's actual dimensions are calculated. This measurement method, which combines image and distance information, provides more accurate dimensional measurement results.

[0121] In summary, combining laser rangefinders or depth sensors for dimensional measurement provides accurate results and offers advantages such as speed, adaptability, and non-contact measurement, bringing significant benefits to the control system of automated appearance inspection robots.

[0122] As one embodiment of the present invention, the sorting container management module intelligently selects the appropriate container type based on the weight attributes of the object to avoid damage and overflow.

[0123] Specifically, the sorting container management module obtains the weight information of objects through weight sensors or weighing devices. These sensors or devices are installed in systems such as sorting robotic arms or assembly lines to monitor the weight of objects in real time.

[0124] In practical applications, the sorting container management module intelligently selects the appropriate container type based on the weight attributes of the object. For example, the maximum load capacity of different container types can be preset and this information can be input into the sorting container management module. When the sorting robot arm sorts the object into the container, the sorting container management module compares the weight of the object with the maximum load capacity of the container and selects the appropriate container type.

[0125] If the weight of an object exceeds the container's maximum load capacity, the sorting container management module will automatically determine and select another container capable of carrying the object's weight to avoid the risk of container damage and overflow. Simultaneously, the module records and calculates the total weight of the objects in each container for subsequent management and processing.

[0126] The sorting container management module acquires the weight information of objects through weight sensors or weighing devices and then compares it with the maximum load capacity of the container. Based on the comparison result, the module intelligently selects the appropriate container type to ensure that the object is safely placed in a container capable of supporting its weight.

[0127] In summary, the intelligent sorting container management module based on object weight attributes avoids container damage and overflow accidents, improves work efficiency, and provides data statistics and management functions, bringing significant benefits to the control system of automated appearance inspection robots.

[0128] As one embodiment of the present invention, the control and decision-making module is based on a reinforcement learning algorithm, which improves sorting efficiency and accuracy through continuous learning and optimization.

[0129] Specifically, the control and decision-making module uses reinforcement learning algorithms for decision-making and control of sorting tasks. Reinforcement learning is a machine learning method that teaches agents (such as robots) how to make optimal decisions to obtain the greatest reward through interaction with the environment.

[0130] In practical applications, the control and decision-making module models the sorting task as a reinforcement learning problem. The agent observes the environmental state, such as the attributes and location information of the objects to be sorted, and selects an appropriate action based on the current state, such as sorting the objects into a designated container. After each action is performed, the agent receives feedback from the environment, including a reward signal and the observed value of the next state. By continuously interacting with the environment and receiving feedback based on reward signals, the agent learns how to make optimal decisions through reinforcement learning algorithms.

[0131] The control and decision-making module is based on reinforcement learning algorithms, learning the optimal sorting strategy through the interaction between the agent and the environment. This module uses reinforcement learning methods such as value functions and policy gradients for decision-making and control.

[0132] In sorting tasks, the agent selects and executes an action based on the current environmental state. After execution, the agent receives feedback from the environment, including reward signals and observations of the next state. The control and decision-making module uses this feedback information to update the policy and value function to improve decision-making and control performance. Through continuous interaction and learning with the environment, the agent gradually improves sorting efficiency and accuracy.

[0133] In summary, the control and decision-making module based on reinforcement learning algorithms improves sorting efficiency and accuracy through continuous learning and optimization. It possesses autonomous learning capabilities and adaptability to complex environments, bringing significant benefits to the control system of automated appearance inspection robots.

[0134] As one embodiment of the present invention, the human-machine interface module supports remote monitoring and operation, and realizes distributed management and data sharing through cloud services.

[0135] Specifically, the human-machine interface module provides a user-friendly interface for remote monitoring and operation of the sorting system. Through this interface, users can view the sorting system's status, data, and video images in real time, and perform remote operation and control.

[0136] In practical applications, the human-machine interface (HMI) module connects to cloud services to achieve distributed management and data sharing. Various components of the sorting system (such as the sorting robot arm, sensors, and controllers) upload data to the cloud service for storage and processing. Users access the cloud service through the HMI module to obtain real-time data and status information.

[0137] Meanwhile, the human-machine interface module also supports remote operation and control of the sorting system. Users send commands to the cloud service through the interface, and the cloud service then transmits the commands to the corresponding equipment in the sorting system for execution. In this way, users can remotely monitor and control the sorting system without directly touching it.

[0138] The human-machine interface (HMI) module connects to cloud services, enabling remote monitoring and operation of the sorting system. Each device in the sorting system uploads data to the cloud service, which then stores and processes this data and provides it to users through the HMI.

[0139] The human-machine interface module obtains real-time status, data, and video images from the sorting system via cloud services. This information is then displayed to the user through the interface, allowing the user to remotely monitor and understand the operation of the sorting system.

[0140] Meanwhile, users send commands to the cloud service through the human-machine interface module, and the cloud service then transmits the commands to the corresponding equipment in the sorting system for execution. In this way, users can remotely operate and control the sorting system, such as starting or stopping the sorting process and adjusting parameter settings.

[0141] In summary, the cloud-based human-machine interface module supports remote monitoring and operation, realizing distributed management and data sharing functions. It offers the convenience of remote monitoring and operation, the flexibility of distributed management, and the advantages of data sharing and analysis, bringing significant benefits to the control system of automated appearance inspection robots.

[0142] As one embodiment of the present invention, the data storage and analysis module uses big data analysis technology to uncover hidden correlations and patterns, providing predictive analysis and intelligent decision support.

[0143] Specifically, the data storage and analysis module collects, stores, and processes the large amounts of data generated by the sorting system, and utilizes big data analytics for data mining and analysis. Through in-depth mining and analysis of this data, it reveals the correlations and patterns, providing predictive analytics and intelligent decision support.

[0144] In practical applications, the data storage and analysis module uses various big data analytics techniques, such as machine learning, data mining, and statistical analysis, to process and analyze the data generated by the sorting system. These techniques extract features, discover patterns, build models, and provide prediction and decision support from the data.

[0145] The data storage and analysis module utilizes big data analytics to mine and analyze the data generated by the sorting system. By collecting, storing, and processing large amounts of data, and applying methods such as machine learning and data mining, the module reveals the correlations and patterns in the data, thereby providing predictive analysis and intelligent decision support.

[0146] The workflow of the data storage and analysis module includes steps such as data collection and storage, data cleaning and preprocessing, feature extraction and selection, model building and training, and predictive analytics and intelligent decision-making. Through the combination of these steps, the module extracts valuable information from the data to provide prediction and decision support.

[0147] In summary, the data storage and analysis module based on big data analytics technology provides predictive analysis and intelligent decision support by uncovering hidden correlations and patterns. It offers predictive analysis and intelligent decision-making capabilities, insights into discovering correlations and patterns, and the advantages of data-driven optimization, bringing significant benefits to the control system of automated appearance inspection robots.

[0148] The automated appearance inspection method described in Example 1 is applicable in this example.

[0149] The automated appearance inspection device described in this embodiment is used to perform the automated appearance inspection method, such as... Figure 7 As shown, it specifically includes:

[0150] In step 601, the image acquisition module is configured and started. Specifically, the image acquisition device on the robot is configured, and the module is started to acquire image data of the logistics packaging box. This is achieved through a camera, sensor, or other suitable device.

[0151] In step 602, a trained deep learning model is used for object recognition. Specifically, the acquired image data is input into the trained deep learning model to identify objects in the image. This model uses deep learning algorithms in the field of computer vision, such as convolutional neural networks and object detection algorithms. The output includes the object's category label and confidence score.

[0152] In step 603, the size information of the object is extracted. Specifically, based on various computer vision techniques, the size information of the identified object is extracted. This includes using image processing and computational geometry techniques, such as edge detection, contour extraction, and shape analysis, to obtain features such as the object's size, area, and perimeter.

[0153] In step 604, the filling status, weight, and size requirements of the sorting containers are monitored in real time. Specifically, sensors, weighing sensors, and other relevant equipment are used to monitor the filling status, weight, and size requirements of the sorting containers in real time. This information is used to determine the location and type of containers to meet sorting requirements.

[0154] In step 605, intelligent algorithms are used to make sorting decisions. Specifically, based on the results of object recognition and size measurement, intelligent algorithms are used to make sorting decisions. This includes optimization algorithms, rule engines, machine learning algorithms, etc., to determine the optimal sorting strategy to achieve high efficiency and accuracy.

[0155] In step 606, an intuitive human-machine interface is provided. Specifically, an intuitive human-machine interface is provided to the operator, allowing real-time information display and operation commands to be received through touchscreen or voice interaction. This interface displays object recognition results, size information, sorting status, etc., while allowing the operator to control and interact with the robot.

[0156] In step 607, the collected data is stored and analyzed and optimized. Specifically, the collected image data, object recognition results, size information, and other relevant data are stored. This data is used for subsequent analysis and optimization, such as system performance evaluation, anomaly detection, and decision optimization.

[0157] Through the above steps, this automated appearance inspection method achieves automated identification, size measurement, and sorting decisions for logistics packaging boxes. It combines image processing, deep learning, intelligent algorithms, and human-computer interaction technologies to improve sorting efficiency and accuracy, and provides real-time information display and operation command reception capabilities. The stored and analyzed data is also used to further optimize system performance and decision-making strategies.

[0158] As one embodiment of the present invention, object recognition uses a multi-level neural network structure to improve the recognition capability in complex scenes and under changing lighting conditions.

[0159] Specifically, object recognition uses multi-layered neural network structures, such as deep convolutional neural networks (CNNs). This network structure consists of multiple convolutional layers, pooling layers, and fully connected layers, which effectively extract features from the input image and perform object classification and recognition.

[0160] In practical applications, the multi-level neural network structure for object recognition has the following characteristics:

[0161] Multi-level feature extraction: Multi-level neural networks extract multi-scale, multi-level features from input images through layer-by-layer convolution and pooling operations. This allows the network to capture object features at different levels, from low-level edges and textures to high-level shapes and structures, improving recognition capabilities.

[0162] Contextual Information Utilization: Multi-layered neural networks utilize local and global contextual information when processing images. Through convolution and pooling operations, the network acquires contextual information surrounding objects at each layer, thereby better understanding the object's background and environment. This is of great significance for object recognition in complex scenes and under varying lighting conditions.

[0163] Nonlinear mapping capability: Multi-layered neural networks map input signals to a high-dimensional feature space through nonlinear activation functions (such as ReLU), thereby possessing stronger modeling capabilities. This allows the network to better adapt to complex object shapes, textures, and variations, improving its adaptability to complex scenes and lighting conditions.

[0164] Parameter sharing and weight sharing: Convolutional layers in multi-layered neural networks use parameter sharing and weight sharing to reduce the number of parameters and computational complexity. This makes the network more lightweight and efficient, suitable for object recognition in resource-constrained environments.

[0165] Pre-training and fine-tuning: Multi-layered neural networks improve recognition performance through pre-training and fine-tuning. Pre-training is performed on large-scale datasets to learn general image features. Then, fine-tuning is performed on task-specific datasets to better adapt the network to the specific object recognition task.

[0166] Through these characteristics, multi-level neural network structures improve object recognition capabilities in complex scenes and under varying lighting conditions. They can extract multi-level features from input images and utilize contextual information for classification and recognition. Through parameter and weight sharing, the network exhibits lightweight and high efficiency. Pre-training and fine-tuning further enhance the network's performance and adaptability.

[0167] In automated appearance inspection methods, using multi-level neural network structures for object recognition effectively improves the ability to identify objects in complex scenes and under varying lighting conditions. This helps improve the accuracy and robustness of sorting systems, enabling more efficient object sorting and processing.

[0168] Example 3:

[0169] like Figure 8 The diagram shown is a schematic representation of the architecture of an automated appearance inspection device according to an embodiment of the present invention. This automated appearance inspection device includes one or more processors 21 and a memory 22. Figure 8 Take a processor 21 as an example.

[0170] Processor 21 and memory 22 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0171] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the automated appearance inspection method in Embodiment 1. The processor 21 executes the automated appearance inspection method by running the non-volatile software program and instructions stored in the memory 22.

[0172] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0173] The program instructions / modules are stored in the memory 22 and, when executed by one or more processors 21, perform the automated appearance inspection method described in Embodiment 1.

[0174] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0175] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automated appearance inspection method, characterized in that, include: Use image acquisition equipment to acquire image data of logistics packaging boxes; The image data is used to perform image recognition using an object recognition model to obtain the objects contained in the logistics packaging box; The object is extracted from the image data and analyzed to obtain the object's size information; Sensors are used to detect the filling status of each sorting container; Based on the object's size information and the filling status of each sorting container, a preset algorithm is used to determine the object's sorting strategy. This includes: identifying first sorting containers for containing objects of that category based on the object's type; determining whether a first sorting container capable of containing the object exists based on the filling status of each first sorting container and the object's size information; if a first sorting container capable of containing the object exists, sorting the object into that container; if no first sorting container capable of containing the object exists, determining the object's sorting strategy based on the object's own safety factor, the object's influence factor on the external environment, the safety factors of existing objects in each second sorting container, and the influence factors of existing objects in each second sorting container on the external environment. The sorting strategy includes: using the object with the highest self-safety factor among the existing objects in the second sorting container as the self-safety factor of the second sorting container; using the object in the second sorting container with the lowest influence factor on the outside world as the influence factor of the second sorting container; using the condition that the influence factor of the second sorting container is not lower than the self-safety factor of the object and the self-safety factor of the second sorting container is not higher than the influence factor of the object on the outside world as a screening condition to screen each second sorting container to obtain a third sorting container that meets the screening condition; determining whether there is a third sorting container that can accommodate the object based on the filling state of each third sorting container and the size information of the object; if there is a third sorting container that can accommodate the object, then sorting the object into the third sorting container that can accommodate the object. The second sorting container is used to hold other types of objects; the object's own safety factor is the potential loss when the object is damaged. The higher the potential loss when the object is damaged, the higher the object's own safety factor; the object's influence factor on the outside world represents the degree of impact on the outside world when the object is damaged and the possibility of the object being damaged.

2. The automated appearance inspection method according to claim 1, characterized in that, Multiple image acquisition devices are used to acquire image data of logistics packaging boxes from multiple different acquisition angles to obtain the objects contained in multiple image data. The step of extracting and analyzing the object from the image data to obtain the object's size information specifically includes: calculating the consistency of the objects contained in each image data; determining whether the objects contained in multiple image data are the same object based on the consistency; if the objects are the same object, matching the same object contained in multiple image data to obtain a three-dimensional model of the object; and analyzing the three-dimensional model to obtain the object's size information.

3. The automated appearance inspection method according to claim 1, characterized in that, The object recognition model is a deep learning model; The deep learning model is pre-trained using an image dataset to facilitate image recognition using the trained deep learning model.

4. The automated appearance inspection method according to claim 1, characterized in that, A laser rangefinder was also installed at the location of the image acquisition equipment; The laser rangefinder is used to measure the distance between the image acquisition device and the object in the logistics packaging box, so that the object's size information can be obtained by jointly analyzing the distance information and image data.

5. The automated appearance inspection method according to claim 1, characterized in that, The specific steps for using sensors to detect the filling status of each sorting container are as follows: The weight information of the sorting container is detected by a weight sensor, and the weight information is compared with the maximum load capacity of the sorting container to obtain the filling status of the sorting container. or, The size of the space occupied in the sorting container is scanned using a laser scanner, and the size of the occupied space is compared with the size of the sorting container itself to obtain the filling status of the sorting container.

6. The automated appearance inspection method according to claim 1, characterized in that, When multiple first sorting containers capable of holding the object exist, the process of sorting the object into the first sorting containers capable of holding the object specifically includes: Calculate the capacity coefficient of each first sorting container that can hold the object, and sort the object into the first sorting container with the highest capacity coefficient; Wherein, the tolerance coefficient ;in, and For preset coefficients, For the object's size information, This indicates the filling status of the first sorting container. This is the distance between the object and the first sorting container.

7. An automated appearance inspection device for implementing the automated appearance inspection method as described in any one of claims 1 to 6, characterized in that, It includes an image acquisition module, an object recognition module, a size measurement module, a sorting container management module, and a control and decision-making module; The image acquisition module is used to acquire image data of logistics packaging boxes using image acquisition equipment; The size measurement module is used to perform image recognition on the image data using an object recognition model to obtain the objects contained in the logistics packaging box; and to extract and analyze the objects from the image data to obtain the size information of the objects. The sorting container management module is used to detect the filling status of each sorting container using sensors; The control and decision module is used to determine the sorting strategy for the object based on the object's size information and the filling status of each sorting container using a preset algorithm.

8. An automated appearance inspection device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the automated appearance inspection method according to any one of claims 1-6.

Images