Self-adaptive sorting system and method based on machine vision

Abstract

The invention relates to the technical field of machine vision, in particular to a self-adaptive sorting system and method based on machine vision. Parcel image data on the sorting line is collected through an image collection device, and the image data is analyzed and recognized in combination with a machine vision method, so that real-time parcel information is obtained; based on the real-time package information, the real-time running state of the sorting line is analyzed and obtained, and a self-adaptive control decision of the sorting line is executed according to the real-time running state; and according to the self-adaptive control decision result, the sorting line is controlled to execute self-adaptive sorting treatment. According to the invention, intelligent and automatic operation of the sorting system can be realized.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to an adaptive sorting system and method based on machine vision. Background Technology

[0002] Existing sorting technologies mainly consist of manual sorting and fixed-program mechanical sorting. The former relies on manual identification of package information and sorting, while the latter mechanically executes sorting actions according to preset fixed parameters. Problems with existing technologies include low efficiency, high error rate, high cost, and difficulty in handling large-scale demands with manual sorting; fixed-program sorting cannot dynamically adapt to the sorting line status, easily leading to congestion or idle runs; weak identification and processing of abnormal packages; and insufficient overall sorting efficiency and stability. Summary of the Invention

[0003] This invention addresses the technical problems of low efficiency, high error rate, high cost, and difficulty in meeting large-scale demands in existing manual sorting technologies by providing an adaptive sorting system and method based on machine vision.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides an adaptive sorting system based on machine vision, comprising: a real-time package information acquisition module, used to acquire package image data on the sorting line through an image acquisition device, and analyze and identify the image data using machine vision methods to acquire real-time package information, wherein the real-time package information includes at least contour information, location information, and identity information; an adaptive control decision execution module, used to analyze and acquire the real-time operating status of the sorting line based on the real-time package information, and execute adaptive control decisions for the sorting line according to the real-time operating status, wherein the adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions; and an adaptive sorting disposal module, used to control the sorting line to perform adaptive sorting disposal according to the adaptive control decision results.

[0005] Optionally, image data of packages on the sorting line is acquired using an image acquisition device, and the image data is analyzed and identified using machine vision methods to obtain real-time package information. This includes: acquiring a trigger signal of the package flow on the sorting line using a trigger sensor; activating the image acquisition device to acquire package image data of the target area in real time based on the trigger signal; performing synchronous preprocessing on the trigger signal and the package image data, wherein the preprocessing includes at least point cloud stitching, noise filtering, and coordinate unification; and inputting the preprocessed trigger signal and the package image data into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package.

[0006] The process involves inputting the preprocessed trigger signal and the package image data into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package. This includes: performing edge detection on the package image data using the target detection machine vision model and iteratively selecting and determining the minimum bounding rectangle for each package; extracting the diagonal coordinates and diagonal direction based on the determined minimum bounding rectangle, and extracting the package height based on the trigger height of the trigger signal, merging the diagonal coordinates and diagonal direction to output the contour information; extracting the center point coordinates of the minimum bounding rectangle and outputting the position information; and defining a unique tracking ID for each package according to the inference order of the target detection machine vision model and outputting it as the identity information.

[0007] Optionally, the image acquisition device can be either a 2D vision sensor or a 3D vision sensor, and the trigger sensing device can be either a photoelectric sensor or a 2D vision sensor.

[0008] Optionally, based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions for the sorting line are executed according to the real-time operating status. The adaptive control decisions include adaptive package feeding decisions and abnormal package identification decisions, including: obtaining the real-time operating status of the sorting line based on the identity information and location information in the real-time package information, wherein the real-time operating status includes loop state parameters and sorting processing efficiency, and the loop state parameters are used to characterize the idle state of downstream conveying equipment; making a decision based on a preset threshold range and the loop state parameters; if the loop state parameters do not meet the threshold range, obtaining the sorting target efficiency set for the sorting line; and calculating package interval adjustment parameters based on the sorting processing efficiency and the sorting target efficiency, combined with a preset package feeding decision model, and outputting the adaptive package feeding decision result.

[0009] Optionally, based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions for the sorting line are executed according to the real-time operating status. The adaptive control decisions include adaptive package feeding decisions and abnormal package identification decisions, and further include: calculating the most favorable throughput time based on the adaptive package feeding decision results, and generating a first steady-state time threshold by combining the results with a preset threshold multiplier; updating the package dwell time of each package in the target area in real time, and marking packages with dwell times greater than the first steady-state time threshold as suspected abnormal packages; obtaining the shape and size information of multiple suspected abnormal packages, and matching them to determine corresponding preset package classes, wherein the preset package classes are associated with and stored typical throughput times; calculating a second adaptive time threshold for each suspected abnormal package by combining the typical throughput time with the preset threshold multiplier, and performing secondary screening on multiple suspected abnormal packages to determine multiple abnormal packages; and outputting the real-time package information of multiple abnormal packages as the abnormal package identification decision result.

[0010] The process involves acquiring shape and size information of multiple suspected abnormal packages and matching them to determine corresponding preset package categories. Each preset package category is associated with a typical transit time. The steps include: based on the identity information of the suspected abnormal package, backtracking and extracting the corresponding diagonal coordinates and package height as the shape and size information; matching the shape and size information with a predefined package category database to determine the preset package category to which the suspected abnormal package belongs. The package category database includes multiple preset package categories obtained by clustering historical sorting data, and each preset package category is associated with a typical transit time; and reading the typical transit time associated with the preset package category from the package category database.

[0011] Optionally, based on the adaptive control decision results, the sorting line is controlled to perform adaptive sorting, including: based on the adaptive package feeding decision results, controlling the timing of the sorting line's actions to adaptively adjust the sorting spacing and package feeding frequency; based on the abnormal package identification decision results, locating multiple abnormal packages, and triggering an automatic recovery mechanism by combining the real-time package information of the multiple abnormal packages, and performing automatic recovery; continuously monitoring multiple abnormal packages, and if the automatic recovery times out and the abnormal package has not yet left the target area, triggering a manual intervention warning and generating an operation blocking command.

[0012] The automatic recovery mechanism includes: extracting a target recovery strategy from a pre-built prior recovery strategy library based on the real-time package information; The prior recovery strategy library is constructed based on prior simulation analysis results and historical recovery records, and the target recovery strategy includes at least one of micro-movement belt, local air blowing, and robotic arm light touch.

[0013] Secondly, the present invention provides an adaptive sorting method based on machine vision, comprising: The image acquisition device collects package image data on the sorting line, and the image data is analyzed and identified using machine vision methods to obtain real-time package information. The real-time package information includes at least outline information, location information, and identity information. Based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions of the sorting line are executed according to the real-time operating status. The adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions. Based on the adaptive control decision results, the sorting line is controlled to perform adaptive sorting and processing.

[0014] By implementing this invention, it is possible to acquire package image data on the sorting line through an image acquisition device, and analyze and identify the image data using machine vision methods to obtain real-time package information. This real-time package information includes at least outline information, location information, and identity information. The use of synchronized processing of machine vision and trigger signals significantly improves the accuracy of information acquisition, avoiding subjective errors from manual identification. Simultaneously, unique package ID tracking ensures process traceability, and optimized data quality in the preprocessing stage further improves model inference efficiency, ensuring real-time information acquisition. This allows for dynamic adaptation to the sorting line's operating status, effectively preventing package congestion or equipment idling, increasing the overall throughput of the sorting line, and accurately identifying abnormal packages to reduce interference with the sorting process and lower the risk of missorting or omissions. The decision-making method combining historical data and preset models also makes decision-making more scientific, eliminating the need for manual parameter adjustments.

[0015] By implementing this invention, it is possible to analyze and obtain the real-time operating status of the sorting line based on the real-time package information, and execute adaptive control decisions for the sorting line according to the real-time operating status. The adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions, realizing a closed loop between decision-making and execution, ensuring the effective implementation of control strategies, improving the automation level of the sorting process, reducing the frequency of manual intervention through automatic recovery mechanisms, and ensuring that abnormal problems can be handled in a timely manner through manual early warning, thus balancing sorting efficiency and operational reliability. At the same time, it can also adapt to different decision-making scenarios and enhance the sorting system's ability to cope with complex package situations.

[0016] By implementing this invention, it is possible to control the sorting line to perform adaptive sorting processing based on the results of adaptive control decisions.

[0017] In summary, by implementing this invention, the intelligent and automated operation of the sorting system can be achieved. This not only improves the accuracy and real-time performance of package information identification, but also dynamically adapts to the operating status of the sorting line and differences in packages, efficiently handling both normal and abnormal package sorting needs. Ultimately, it reduces labor costs, decreases sorting error rates, improves sorting efficiency and system stability, and is suitable for large-scale, diverse package sorting scenarios. Attached Figure Description

[0018] Figure 1 A schematic diagram of the structure of an adaptive sorting system based on machine vision provided by the present invention; Figure 2 This is a flowchart illustrating an adaptive sorting method based on machine vision provided by the present invention.

[0019] In the attached diagram, the components represented by each number are as follows: The module includes a real-time package information acquisition module 11, an adaptive control decision execution module 12, and an adaptive sorting and disposal module 13. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0023] Example 1, as Figure 1 As shown, this embodiment of the invention provides an adaptive sorting system based on machine vision, comprising: The real-time package information acquisition module 11 is used to acquire package image data on the sorting line through an image acquisition device, and to analyze and identify the image data in combination with machine vision methods to obtain real-time package information, wherein the real-time package information includes at least outline information, location information and identity information. The adaptive control decision execution module 12 is used to analyze and obtain the real-time operating status of the sorting line based on the real-time package information, and execute the adaptive control decision of the sorting line according to the real-time operating status. The adaptive control decision includes adaptive package supply decision and abnormal package identification decision. The adaptive sorting and processing module 13 is used to control the sorting line to perform adaptive sorting and processing based on the adaptive control decision results.

[0024] In the real-time package information acquisition module 11 of this application embodiment, package image data on the sorting line is acquired by an image acquisition device, and the image data is analyzed and identified by machine vision methods to obtain real-time package information, including: Trigger signals from the parcel flow are collected at the sorting line by triggering sensors. Based on the trigger signal, the image acquisition device is activated to acquire package image data of the target area in real time; The trigger signal and the package image data are preprocessed synchronously, wherein the preprocessing includes at least point cloud stitching, noise filtering and coordinate unification; The preprocessed trigger signal and the package image data are input into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package.

[0025] In the real-time package information acquisition module 11 of this application embodiment, the image acquisition device is any one of a 2D vision sensor and a 3D vision sensor, and the trigger sensing device is any one of a photoelectric sensor and a 2D vision sensor.

[0026] In this embodiment, the function of the real-time package information acquisition module 11 is to accurately and in real-time acquire key information of each package on the sorting line, providing data support for subsequent adaptive package supply control and abnormal package handling. Specifically, this includes clearly identifying the package outline, location, and identity to ensure dynamic matching between the front-end package supply and the downstream loop sorting machine, while also laying the foundation for abnormal package identification and tracking.

[0027] To achieve the above objectives, it is first necessary to use a trigger sensor to collect trigger signals from the package flow on the sorting line.

[0028] This involves deploying trigger sensors at key locations on the sorting line. These sensors can be either photoelectric sensors or 2D vision sensors. When a package passes through the detection range of the device, it is triggered and generates an electrical or digital signal to mark that the package has arrived at the collection area.

[0029] For example, a photoelectric sensor can be selected as the triggering device and installed 1 meter before the loop crossover enters each package supply area. The sensor detection distance is set to 0.5 meters, and the response time is ≤10ms. When a package passes by and blocks the sensor's light, a high-level trigger signal is immediately generated. If a 2D vision sensor is selected as the triggering device, the trigger signal is generated by recognizing whether the outline of the package appears in the image, and the detection frame rate is set to 30 frames / second.

[0030] Next, based on the trigger signal, the image acquisition device is activated to acquire real-time image data of the package in the target area.

[0031] The trigger signal is transmitted to the image acquisition device via the communication line, serving as a start command to activate the device, which then immediately begins continuous image acquisition of packages in the target area. The image acquisition device can be either a 2D vision sensor or a 3D vision sensor, which can be selected based on the accuracy requirements of the sorting scenario.

[0032] For example, select a 3D vision sensor with a resolution higher than 2 megapixels and install it above each belt set and in the single-separation area. If the single-separation length exceeds 5 meters, install two 3D vision sensors. After the trigger signal arrives, the sensor acquires images at a frame rate of 60 frames per second. The target area covers the entire length of a single belt and the junction of adjacent belts. The acquisition range covers the belt width laterally, for example, 1.2 meters, and the possible height range of the package vertically, such as 0-0.5 meters. If a 2D vision sensor is selected, the acquisition resolution is set to 1920×1080 pixels, and the exposure time is adjusted to 1 / 1000 second to avoid motion blur.

[0033] Then, the trigger signal and the package image data are preprocessed synchronously, wherein the preprocessing includes at least point cloud stitching, noise filtering and coordinate unification.

[0034] First, the trigger signal and image data are aligned using timestamps to ensure that each image frame corresponds to a specific package trigger event. Then, point cloud stitching, noise filtering, and coordinate unification operations are performed sequentially to improve data quality and consistency.

[0035] For example, point cloud stitching uses data collected from two 3D vision sensors. A unified coordinate association is established through chessboard calibration, and the two images are stitched into a complete single-region image with a stitching error of ≤2mm. Noise filtering uses a Gaussian filtering algorithm with a filter kernel size of 3×3 to remove random noise and sensor noise from the image. Coordinate unification is achieved by converting the detection coordinates of all sensors into the system's global coordinate system, with the X-axis along the belt conveyor direction, the Y-axis perpendicular to the belt, and the Z-axis in the height direction, with a coordinate accuracy of ±0.5mm.

[0036] Furthermore, the preprocessed trigger signal and the package image data need to be input into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package.

[0037] In the real-time package information acquisition module 11 of this application embodiment, the preprocessed trigger signal and the package image data are input into a pre-trained target detection machine vision model for real-time inference to acquire the real-time package information for each package, including: The target detection machine vision model is used to perform edge detection on the package image data and iteratively selects and determines the minimum bounding rectangle of each package. Based on the determined minimum bounding rectangle, the coordinates of the diagonal points and the direction of the diagonal are extracted, and the wrapping height is extracted based on the trigger height of the trigger signal. The wrapping height is then combined with the coordinates of the diagonal points and the direction of the diagonal to output the contour information. Extract the coordinates of the center point of the minimum bounding rectangle and output them as the position information; Based on the reasoning order of the target detection machine vision model, a unique tracking ID for each package is defined and output as the identity information.

[0038] In the real-time package information acquisition module 11 of this application embodiment, the purpose of the above steps is to accurately extract the outline information, location information and identity information of each package from the preprocessed trigger signal and package image data, so as to provide standardized and traceable core data support for the identification decision of subsequent steps and ensure that the system can achieve accurate control over each package.

[0039] To achieve the above objectives, firstly, the target detection machine vision model is used to perform edge detection on the package image data, and then iteratively selects and determines the minimum bounding rectangle of each package.

[0040] This involves using a machine vision model for object detection to perform edge detection algorithms on the preprocessed package image data, identifying the pixel boundaries between the package and the background, and then iteratively calculating to select the smallest rectangle that can completely surround a single package, ensuring that the rectangle does not exceed the actual area of ​​the package and has the smallest possible area.

[0041] For example, edge detection uses the Canny algorithm with thresholds set to a low threshold of 50 and a high threshold of 150. During the iterative selection process, the pixel error tolerance is set to 2 pixels to ensure that the rectangle can accurately fit the outline of the package. For example, if the actual size of the package is 300mm long and 200mm wide, the size error of the determined minimum bounding rectangle is ≤5mm.

[0042] The target detection machine vision model collects a large amount of image data of package sorting scenarios of different sizes, shapes and materials, and annotates key information such as the outline and position of the packages to build a training dataset. After data augmentation, it is trained based on mature deep learning frameworks such as YOLO and SSD, and the model parameters are iteratively optimized to make the detection accuracy and real-time performance meet the sorting requirements. Finally, it is deployed and used after being verified by testing.

[0043] Next, based on the determined minimum bounding rectangle, the coordinates of the diagonal points and the direction of the diagonal line need to be extracted, and the wrapping height needs to be extracted based on the trigger height of the trigger signal. These are then combined with the coordinates of the diagonal points and the direction of the diagonal line to output the contour information.

[0044] Based on the determined minimum bounding rectangle, the coordinates of the four corner points of the rectangle and the direction of the diagonal extension are read. At the same time, the height data when the package is triggered is extracted from the trigger signal. These three types of data are integrated and output as the outline information of the package.

[0045] For example, the coordinates of the diagonal points are based on the system's global coordinate system. For instance, the coordinates of diagonal point 1 are X=150mm, Y=80mm, and Z=0mm, and the coordinates of diagonal point 2 are X=450mm, Y=280mm, and Z=0mm. The angle between the diagonal direction and the X-axis (belt conveying direction) is 15°. The package height extracted by the trigger signal is 120mm. The three are combined to form the complete outline information of the package.

[0046] Then, the coordinates of the center point of the smallest bounding rectangle need to be extracted and output as the position information.

[0047] That is, the coordinates of the center point of the smallest bounding rectangle are obtained through geometric calculation. These coordinates are directly output as the location information of the package, which is used to locate the real-time position of the package on the sorting line.

[0048] For example, based on the above diagonal coordinates, the center point coordinates are X=300mm, Y=180mm, Z=60mm, with a coordinate accuracy of ±0.5mm, ensuring that the specific position of the package in the belt conveyor direction, lateral direction, and height direction can be accurately reflected.

[0049] Finally, based on the reasoning order of the target detection machine vision model, a unique tracking ID for each package needs to be defined and output as the identity information.

[0050] That is, according to the reasoning and recognition order of the packages by the target detection machine vision model, a unique tracking ID is assigned to each detected package. This ID remains unchanged during the subsequent movement of the package and is used to continuously associate various information about the package.

[0051] For example, the object detection machine vision model assigns IDs to packages in the order they are detected. For instance, the ID of the first detected package is 0001, the second is 0002, and the ID remains unchanged even if the package moves in subsequent frames, ensuring full tracking of a single package.

[0052] In the adaptive control decision execution module 12 of this application embodiment, based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions for the sorting line are executed according to the real-time operating status. The adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions, including: Based on the identity information and location information in the real-time package information, the real-time operating status of the sorting line is obtained, wherein the real-time operating status includes loop status parameters and sorting efficiency, and the loop status parameters are used to characterize the idle status of downstream conveying equipment. Decisions are made based on a preset threshold range and the loop state parameters. If the loop state parameters do not meet the threshold range, the sorting target efficiency set by the sorting line is obtained. Based on the sorting efficiency and the sorting target efficiency, the package interval adjustment parameters are calculated using a preset package supply decision model, and the output is an adaptive package supply decision result.

[0053] In the adaptive control decision execution module 12 of this application embodiment, the purpose of the above steps is to analyze the sorting line operation status through real-time package information, formulate adaptive package supply decisions, so that the front-end package supply rhythm is accurately matched with the downstream loop sorting capacity, avoid efficiency idleness caused by package overload congestion or insufficient package supply, and maximize the overall throughput of the sorting system.

[0054] To achieve the above objectives, it is first necessary to obtain the real-time operating status of the sorting line based on the identity information and location information in the real-time package information. The real-time operating status includes loop status parameters and sorting efficiency. The loop status parameters are used to characterize the idle status of downstream conveying equipment.

[0055] This means that by relying on the identity and location information of each package, the flow trajectory of the package on the sorting line is continuously tracked, and the sorting efficiency is obtained by counting the number of packages sorted per unit time; at the same time, the operation data of the downstream conveying equipment is collected through the loop status monitoring unit, and the loop status parameters are calculated to represent the idle state.

[0056] For example, the identity information is the unique tracking ID of the package, such as 0003, and the location information is the center point coordinates X=500mm, Y=200mm, Z=70mm. Based on this information, the number of packages sorted in 1 hour is 8000, and the sorting efficiency is 8000 packages / hour. The loop status parameter uses the empty vehicle rate. There are 100 carts in the downstream loop. If 30 carts without packages are detected in real time, the empty vehicle rate is 30%.

[0057] Next, a decision needs to be made based on the preset threshold range and the loop state parameters. If the loop state parameters do not meet the threshold range, the sorting target efficiency set by the sorting line is obtained.

[0058] That is, a threshold range of the preset loop status parameters is used to compare the real-time calculated loop status parameters with this range. If the range is exceeded, the preset sorting target efficiency of the sorting line is retrieved.

[0059] For example, if the preset empty rate threshold range is 30%-70%, and the real-time empty rate is 25%, which does not meet the threshold range, then the sorting target efficiency set by the sorting line is obtained. If there are 4 packing stations in the packing area, and the target efficiency of each packing station is 2200 pieces / hour, then the sorting target efficiency is 8800 pieces / hour.

[0060] Then, based on the sorting efficiency and the sorting target efficiency, the package interval adjustment parameters need to be calculated and obtained in combination with the preset package supply decision model, and the output is the adaptive package supply decision result.

[0061] The sorting efficiency and sorting target efficiency are input into a preset package supply decision model. The model combines logic such as the single-item separation theoretical efficiency calculation formula to calculate the package interval adjustment parameters, which are then output as the adaptive package supply decision result.

[0062] For example, if the sorting efficiency is 8000 pieces / hour and the target efficiency is 8800 pieces / hour, there is an efficiency gap of 800 pieces / hour. The package supply decision model references the theoretical efficiency formula: Theoretical efficiency = Maximum linear velocity per unit / (Separation distance + Average package length) × 3600. To match the target efficiency, the required separation distance is calculated based on current measured data. The measured current linear velocity per unit is 1.8 m / s, and the average package length is 0.3 m. The theoretical separation distance should satisfy: 1.8 / (Separation distance + 0.3) × 3600 = 8800, which gives a separation distance of approximately 0.44 m. The current separation distance is 0.8 m, therefore the separation distance needs to be adjusted to 0.44 m, i.e., reduced by 360 mm. This adjustment value is the package interval adjustment parameter.

[0063] In the adaptive control decision execution module 12 of this application embodiment, based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions for the sorting line are executed according to the real-time operating status. The adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions, and further include: The adaptive package supply decision result is used to calculate the most favorable transit time and, combined with a preset threshold multiplier, to generate the first steady-state time threshold. The package dwell time of each package in the target area is updated in real time, and packages whose dwell time is greater than the first steady state time threshold are marked as suspected abnormal packages. Obtain the shape and size information of multiple suspected abnormal packages, and match and determine the corresponding preset package class, wherein the preset package class is associated with and stored with typical transit time; By combining the typical throughput time with the preset threshold ratio, a second adaptive time threshold is calculated for each of the suspected abnormal packages, and multiple suspected abnormal packages are traversed for secondary screening to determine multiple abnormal packages. The real-time package information of multiple abnormal packages is output as the abnormal package identification decision result.

[0064] In the adaptive control decision execution module 12 of this application embodiment, the purpose of the above steps is to accurately identify abnormal packages through dual threshold screening, avoid misjudgment by a single threshold, ensure that only packages that are truly stuck or cannot be transported normally are processed, ensure the continuous and stable operation of the sorting line, and reduce unplanned downtime and ineffective manual intervention.

[0065] Specifically, to achieve the above objectives, the most favorable transit time needs to be obtained by adaptive packet supply decision calculation, and a first steady-state time threshold needs to be generated by combining the preset threshold multiplier.

[0066] Based on parameters such as package interval and delivery speed in the adaptive package supply decision results, the most favorable transit time of the package in the target area is calculated, and then multiplied by a preset threshold multiplier to obtain the first steady-state time threshold.

[0067] Assuming a scenario where the adaptive package feeding decision, calculated using the aforementioned method, determines a package separation distance of 0.6m and a single-disengagement conveyor belt speed of 1.5m / s, then the optimal passage time = separation distance / conveyor belt speed = 0.4 seconds. With a preset threshold multiplier of 5, the first steady-state time threshold = 0.4 seconds × 5 = 2 seconds.

[0068] Next, it is necessary to update the package dwell time of each package in the target area in real time, and mark packages whose dwell time is greater than the first stationary time threshold as suspected abnormal packages.

[0069] This means that by using the identity and location information of each package, the time each package stays in the target area in real time, i.e., the package dwell time, is tracked, and packages whose dwell time exceeds the first steady-state time threshold are marked as suspected abnormal packages.

[0070] For example, if the location information of package ID 0008 is continuously displayed in the 300-400mm range of the X-axis of the single-displacement area, and the cumulative dwell time reaches 2.5 seconds, exceeding the first stationary time threshold of 2 seconds, the package is marked as a suspected abnormal package.

[0071] Next, it is necessary to obtain the shape and size information of multiple suspected abnormal packages and match and determine the corresponding preset package class, wherein the preset package class is associated with a typical pass time.

[0072] In the adaptive control decision execution module 12 of this application embodiment, the shape and size information of multiple suspected abnormal packages is obtained, and the corresponding preset package class is determined by matching. The preset package class is associated with and stored with typical transit times, including: Based on the identity information of the suspected abnormal package, the corresponding diagonal coordinates and package height are traced back and extracted as the shape and size information; The shape and size information is matched with a predefined package category database to determine the preset package category to which the suspected abnormal package belongs. The package category database includes multiple preset package categories obtained by clustering historical sorting data, and the preset package category is associated with the typical transit time. Read the typical transit time associated with the preset package category from the package category database.

[0073] In this embodiment of the application, the purpose of the above steps is to extract the shape and size information of suspected abnormal packages and match them with the corresponding preset package type to obtain the typical transit time of such packages, so as to provide data support for subsequent calculation of the second adaptive time threshold and accurate screening of truly abnormal packages, and avoid misjudgment caused by differences in package type.

[0074] Specifically, to achieve the above objective, it is first necessary to backtrack and extract the corresponding diagonal coordinates and package height as the shape and size information based on the identity information of the suspected abnormal package.

[0075] That is, by using the unique identification information of suspected abnormal packages, we can link and trace the diagonal coordinates and package height data in the previous real-time package information, and use this data as the shape and size information of the package.

[0076] For example, the suspected abnormal package has the ID0012. By tracing its historical data, the diagonal coordinates are extracted as X1=200mm, Y1=100mm, X2=500mm, and Y2=250mm, and the package height is 150mm. These data together constitute the shape and size information of the package.

[0077] Then, the shape and size information is matched with a predefined package category database to determine the preset package category to which the suspected abnormal package belongs. The package category database includes multiple preset package categories obtained by clustering historical sorting data, and the preset package category is associated with the typical transit time.

[0078] The extracted shape and size information is input into a predefined package category database. Data comparison is used to determine the preset package category to which a suspected abnormal package belongs. The package category database is constructed by clustering historical sorting data and contains multiple preset package categories, each associated with a typical throughput time.

[0079] For example, the package category database contains four preset package categories: flat, cube, long and narrow, and irregular. The shape and size information of ID0012 is calculated to be 300mm long, 150mm wide, and 150mm high, with an aspect ratio close to 1. This matches the clustering characteristics of cube-shaped packages, thus determining that it belongs to the cube-shaped preset package category.

[0080] Next, the typical transit time associated with the preset package category is read from the package category database.

[0081] That is, based on the determined preset package category, the typical transit time associated with that category is searched and read from the package category database.

[0082] For example, in the parcel category database, the typical transit time for cube-shaped parcels is 0.5 seconds based on historical data. This data can be directly read as the typical transit time for parcel ID0012.

[0083] Furthermore, it is necessary to combine the typical throughput time with a preset threshold ratio to calculate and obtain a second adaptive time threshold for each of the suspected abnormal packages, and then traverse multiple suspected abnormal packages for secondary screening to determine multiple abnormal packages.

[0084] The typical transit time for each suspected abnormal package is multiplied by a preset threshold multiplier to obtain a second adaptive time threshold specific to that package. The threshold multiplier is consistent with that used when generating the first steady-state time threshold.

[0085] For example, the suspected abnormal package ID0015 belongs to the preset package category of long and narrow, with a typical transit time of 0.8 seconds and a preset threshold multiplier of 5. The second adaptive time threshold is 0.8 seconds × 5 = 4 seconds. Another suspected abnormal package ID0016 belongs to the flat category, with a typical transit time of 0.4 seconds and a second adaptive time threshold of 0.4 seconds × 5 = 2 seconds.

[0086] Then, all suspected abnormal packages are traversed, and the real-time dwell time of each package is compared with the corresponding second adaptive time threshold. If the dwell time exceeds the threshold, it is determined to be the final abnormal package.

[0087] For example, the real-time dwell time of package ID0015 has reached 4.3 seconds, exceeding its second adaptive time threshold of 4 seconds, and it is determined to be an abnormal package; the real-time dwell time of package ID0016 is 1.8 seconds, which does not exceed its second adaptive time threshold of 2 seconds, so the abnormal judgment is excluded and it is restored to a normal package.

[0088] Finally, the real-time package information of multiple abnormal packages needs to be correlated and output as the abnormal package identification decision result.

[0089] The system integrates real-time package information for each identified abnormal package, including identity information, outline information, location information, dwell time, preset package category, and typical passage time, to form a structured abnormal package identification decision result and output it.

[0090] For example, the output result is abnormal package ID0015, the outline information is the diagonal coordinates X1=180mm, Y1=90mm, X2=680mm, Y2=190mm, height 120mm, the position information is X=430mm, Y=140mm, Z=60mm, the dwell time is 4.3 seconds, the preset package type is long strip, and the typical passage time is 0.8 seconds.

[0091] In the adaptive sorting and handling module 13 of this application, the sorting line is controlled to perform adaptive sorting and handling based on the adaptive control decision results, including: Based on the adaptive package feeding decision results, control the timing of the sorting line's actions to adaptively adjust the sorting spacing and package feeding frequency; Based on the abnormal package identification decision, multiple abnormal packages are located, and the automatic recovery mechanism is triggered by combining the real-time package information of the multiple abnormal packages to perform automatic recovery; Continuously monitor multiple abnormal packages. If the automatic recovery timeout expires and the abnormal package has not left the target area, a manual intervention warning will be triggered, and an operation blocking command will be generated.

[0092] The automatic recovery mechanism in the adaptive sorting and handling module 13 of this application includes: Based on the real-time package information, the target recovery strategy is extracted from the pre-built prior recovery strategy library; The prior recovery strategy library is constructed based on prior simulation analysis results and historical recovery records, and the target recovery strategy includes at least one of micro-movement belt, local air blowing, and robotic arm light touch.

[0093] In the adaptive sorting and handling module 13 of this application embodiment, the core purpose of the above steps is to translate the adaptive package supply decision and abnormal package identification decision into specific sorting actions. By dynamically adjusting the package supply rhythm and handling abnormal packages in a targeted manner, the sorting line can achieve efficient collaborative operation, while reducing package damage and unplanned downtime, and lowering the cost of manual intervention.

[0094] To achieve the above objectives, it is first necessary to control the timing of the sorting line's actions based on the adaptive package feeding decision results in order to adaptively adjust the sorting spacing and package feeding frequency.

[0095] That is, it receives the package interval adjustment parameters from the adaptive package feeding decision results, controls the belt action timing of the sorting line execution module, and adjusts the sorting spacing and package feeding frequency by changing the belt start-stop interval and running speed.

[0096] For example, the adaptive package feeding decision results in reducing the separation distance by 150mm. The current sorting distance is 800mm, and the adjusted distance is 650mm. The timing of the single-disengagement conveyor belt is shortened from conveying one package every 3 seconds to conveying one package every 2.2 seconds. The package feeding frequency is increased from 1200 pieces / hour to 1636 pieces / hour, while the single-disengagement conveyor belt speed is maintained at 1.8m / s.

[0097] Next, based on the abnormal package identification decision results, multiple abnormal packages need to be located, and the automatic recovery mechanism needs to be triggered by combining the real-time package information of the multiple abnormal packages to perform automatic recovery.

[0098] Based on real-time package information such as location, outline, and category from the abnormal package identification decision results, a suitable target recovery strategy is matched from the prior recovery strategy library, and the corresponding execution component is controlled to perform automatic recovery operations. The prior recovery strategy library is constructed based on prior simulation analysis results and historical recovery records, and the target recovery strategy includes at least one of the following: micro-motion belt, local air blowing, and robotic arm light touch.

[0099] For example, the location information of the abnormal package ID0020 is X=400mm, Y=200mm, Z=80mm, and the outline information shows that it is a circular package. The preset package category is easy to roll. The target recovery strategy extracted from the prior recovery strategy library is a micro-motion belt. The belt in the area where the package is located is controlled to run forward for 1 second and then reverse for 0.5 seconds at a speed of 0.5m / s, and the cycle is repeated 3 times. The inertia is used to move the package away from the stuck position.

[0100] Finally, multiple abnormal packages need to be continuously monitored. If the automatic recovery times out and the abnormal package has not left the target area, a manual intervention warning will be triggered and an operation blocking command will be generated.

[0101] This involves continuously tracking the location of abnormal packages using a visual recognition unit to determine if they have left the target area. A preset automatic recovery timeout is set; if the abnormal package remains within the timeout period, an audible and visual alarm is immediately triggered to allow for manual intervention. Simultaneously, an operation blocking command is generated to prevent subsequent system actions from affecting the delivery of either abnormal or normal packages.

[0102] For example, if the preset automatic recovery timeout is 20 seconds, and the location information of abnormal package ID0020 remains unchanged within 20 seconds after the micro-motion belt recovery operation, the system immediately activates an audible and visual alarm. The alarm sound is set to 85dB, and the alarm light flashes twice per second. Simultaneously, an operation blocking command is generated to prohibit belt acceleration or spacing adjustments in that area, preventing the compression of abnormal packages or the jamming of normal packages.

[0103] Example 2, as Figure 2 As shown, based on the same inventive concept as the machine vision-based adaptive sorting system provided in Embodiment 1, this embodiment of the invention also provides a machine vision-based adaptive sorting method, including: S100: The image acquisition device acquires package image data on the sorting line, and the image data is analyzed and identified using machine vision methods to obtain real-time package information, wherein the real-time package information includes at least outline information, location information and identity information. S200: Based on the real-time package information, analyze and obtain the real-time operating status of the sorting line, and execute adaptive control decisions for the sorting line according to the real-time operating status, wherein the adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions. S300: Based on the adaptive control decision results, control the sorting line to perform adaptive sorting processing.

[0104] Furthermore, step S100 in this embodiment includes the following execution steps: Trigger signals from the parcel flow are collected at the sorting line by triggering sensors. Based on the trigger signal, the image acquisition device is activated to acquire package image data of the target area in real time; The trigger signal and the package image data are preprocessed synchronously, wherein the preprocessing includes at least point cloud stitching, noise filtering and coordinate unification; The preprocessed trigger signal and the package image data are input into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package.

[0105] The preprocessed trigger signal and the package image data are input into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package, including: The target detection machine vision model is used to perform edge detection on the package image data and iteratively selects and determines the minimum bounding rectangle of each package. Based on the determined minimum bounding rectangle, the coordinates of the diagonal points and the direction of the diagonal are extracted, and the wrapping height is extracted based on the trigger height of the trigger signal. The wrapping height is then combined with the coordinates of the diagonal points and the direction of the diagonal to output the contour information. Extract the coordinates of the center point of the minimum bounding rectangle and output them as the position information; Based on the reasoning order of the target detection machine vision model, a unique tracking ID for each package is defined and output as the identity information.

[0106] Furthermore, step S200 in this embodiment includes the following execution steps: Based on the identity information and location information in the real-time package information, the real-time operating status of the sorting line is obtained, wherein the real-time operating status includes loop status parameters and sorting efficiency, and the loop status parameters are used to characterize the idle status of downstream conveying equipment. Decisions are made based on a preset threshold range and the loop state parameters. If the loop state parameters do not meet the threshold range, the sorting target efficiency set by the sorting line is obtained. Based on the sorting efficiency and the sorting target efficiency, the package interval adjustment parameters are calculated using a preset package supply decision model, and the output is an adaptive package supply decision result.

[0107] The adaptive package supply decision result is used to calculate the most favorable transit time and, combined with a preset threshold multiplier, to generate the first steady-state time threshold. The package dwell time of each package in the target area is updated in real time, and packages whose dwell time is greater than the first steady state time threshold are marked as suspected abnormal packages. Obtain the shape and size information of multiple suspected abnormal packages, and match and determine the corresponding preset package class, wherein the preset package class is associated with and stored with typical transit time; By combining the typical throughput time with the preset threshold ratio, a second adaptive time threshold is calculated for each of the suspected abnormal packages, and multiple suspected abnormal packages are traversed for secondary screening to determine multiple abnormal packages. The real-time package information of multiple abnormal packages is output as the abnormal package identification decision result.

[0108] This involves acquiring shape and size information of multiple suspected abnormal packages and matching them to determine corresponding preset package classes. These preset package classes are associated with and stored with typical transit times, including: Based on the identity information of the suspected abnormal package, the corresponding diagonal coordinates and package height are traced back and extracted as the shape and size information; The shape and size information is matched with a predefined package category database to determine the preset package category to which the suspected abnormal package belongs. The package category database includes multiple preset package categories obtained by clustering historical sorting data, and the preset package category is associated with the typical transit time. Read the typical transit time associated with the preset package category from the package category database.

[0109] Furthermore, step S300 in this embodiment includes the following execution steps: Based on the adaptive package feeding decision results, control the timing of the sorting line's actions to adaptively adjust the sorting spacing and package feeding frequency; Based on the abnormal package identification decision, multiple abnormal packages are located, and the automatic recovery mechanism is triggered by combining the real-time package information of the multiple abnormal packages to perform automatic recovery; Continuously monitor multiple abnormal packages. If the automatic recovery timeout expires and the abnormal package has not left the target area, a manual intervention warning will be triggered, and an operation blocking command will be generated.

[0110] The automatic recovery mechanism includes: Based on the real-time package information, the target recovery strategy is extracted from the pre-built prior recovery strategy library; The prior recovery strategy library is constructed based on prior simulation analysis results and historical recovery records, and the target recovery strategy includes at least one of micro-movement belt, local air blowing, and robotic arm light touch.

[0111] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0112] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Clearly, those skilled in the art can make various alterations and variations to the invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the invention and its equivalents, the invention is also intended to include these modifications and variations.

Claims

1. An adaptive sorting system based on machine vision, characterized in that, include: The real-time package information acquisition module is used to acquire package image data on the sorting line through an image acquisition device, and to analyze and identify the image data in combination with machine vision methods to obtain real-time package information, wherein the real-time package information includes at least outline information, location information and identity information. An adaptive control decision execution module is used to analyze and obtain the real-time operating status of the sorting line based on the real-time package information, and execute adaptive control decisions for the sorting line according to the real-time operating status. The adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions. The adaptive sorting and processing module is used to control the sorting line to perform adaptive sorting and processing based on the results of adaptive control decisions.

2. The adaptive sorting system based on machine vision as described in claim 1, characterized in that, Image data of packages on the sorting line is acquired using an image acquisition device, and the image data is analyzed and identified using machine vision methods to obtain real-time package information, including: Trigger signals from the parcel flow are collected at the sorting line by triggering sensors. Based on the trigger signal, the image acquisition device is activated to acquire package image data of the target area in real time; The trigger signal and the package image data are preprocessed synchronously, wherein the preprocessing includes at least point cloud stitching, noise filtering and coordinate unification; The preprocessed trigger signal and the package image data are input into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package.

3. The adaptive sorting system based on machine vision as described in claim 2, characterized in that, The preprocessed trigger signal and the package image data are input into a pre-trained target detection machine vision model for real-time inference to obtain the real-time package information for each package, including: The target detection machine vision model is used to perform edge detection on the package image data and iteratively selects and determines the minimum bounding rectangle of each package. Based on the determined minimum bounding rectangle, the coordinates of the diagonal points and the direction of the diagonal are extracted, and the wrapping height is extracted based on the trigger height of the trigger signal. The wrapping height is then combined with the coordinates of the diagonal points and the direction of the diagonal to output the contour information. Extract the coordinates of the center point of the minimum bounding rectangle and output them as the position information; Based on the reasoning order of the target detection machine vision model, a unique tracking ID for each package is defined and output as the identity information.

4. The adaptive sorting system based on machine vision as described in claim 1, characterized in that, The image acquisition device can be either a 2D vision sensor or a 3D vision sensor, and the trigger sensing device can be either a photoelectric sensor or a 2D vision sensor.

5. The adaptive sorting system based on machine vision as described in claim 1, characterized in that, Based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions for the sorting line are executed according to the real-time operating status. These adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions, including: Based on the identity information and location information in the real-time package information, the real-time operating status of the sorting line is obtained, wherein the real-time operating status includes loop status parameters and sorting efficiency, and the loop status parameters are used to characterize the idle status of downstream conveying equipment. Decisions are made based on a preset threshold range and the loop state parameters. If the loop state parameters do not meet the threshold range, the sorting target efficiency set by the sorting line is obtained. Based on the sorting efficiency and the sorting target efficiency, the package interval adjustment parameters are calculated using a preset package supply decision model, and the output is an adaptive package supply decision result.

6. The adaptive sorting system based on machine vision as described in claim 1, characterized in that, Based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions for the sorting line are executed according to the real-time operating status. These adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions, and further include: The adaptive package supply decision result is used to calculate the most favorable transit time and, combined with a preset threshold multiplier, to generate the first steady-state time threshold. The package dwell time of each package in the target area is updated in real time, and packages whose dwell time is greater than the first steady state time threshold are marked as suspected abnormal packages. Obtain the shape and size information of multiple suspected abnormal packages, and match and determine the corresponding preset package class, wherein the preset package class is associated with and stored with typical transit time; By combining the typical throughput time with the preset threshold ratio, a second adaptive time threshold is calculated for each of the suspected abnormal packages, and multiple suspected abnormal packages are traversed for secondary screening to determine multiple abnormal packages. The real-time package information of multiple abnormal packages is output as the abnormal package identification decision result.

7. The adaptive sorting system based on machine vision as described in claim 6, characterized in that, Obtain the shape and size information of multiple suspected abnormal packages, and match them to determine the corresponding preset package class. The preset package class is associated with and stored with typical transit times, including: Based on the identity information of the suspected abnormal package, the corresponding diagonal coordinates and package height are traced back and extracted as the shape and size information; The shape and size information is matched with a predefined package category database to determine the preset package category to which the suspected abnormal package belongs. The package category database includes multiple preset package categories obtained by clustering historical sorting data, and the preset package category is associated with the typical transit time. Read the typical transit time associated with the preset package category from the package category database.

8. The adaptive sorting system based on machine vision as described in claim 1, characterized in that, Based on the adaptive control decision results, the sorting line is controlled to perform adaptive sorting processes, including: Based on the adaptive package feeding decision results, control the timing of the sorting line's actions to adaptively adjust the sorting spacing and package feeding frequency; Based on the abnormal package identification decision, multiple abnormal packages are located, and the automatic recovery mechanism is triggered by combining the real-time package information of the multiple abnormal packages to perform automatic recovery; Continuously monitor multiple abnormal packages. If the automatic recovery timeout expires and the abnormal package has not left the target area, a manual intervention warning will be triggered, and an operation blocking command will be generated.

9. The adaptive sorting system based on machine vision as described in claim 8, characterized in that, Automatic recovery mechanisms include: Based on the real-time package information, the target recovery strategy is extracted from the pre-built prior recovery strategy library; The prior recovery strategy library is constructed based on prior simulation analysis results and historical recovery records, and the target recovery strategy includes at least one of micro-movement belt, local air blowing, and robotic arm light touch.

10. An adaptive sorting method based on machine vision, characterized in that, The method can be executed by an adaptive sorting system based on machine vision as described in any one of claims 1-9, comprising: The image acquisition device collects package image data on the sorting line, and the image data is analyzed and identified using machine vision methods to obtain real-time package information. The real-time package information includes at least outline information, location information, and identity information. Based on the real-time package information, the real-time operating status of the sorting line is analyzed and obtained, and adaptive control decisions of the sorting line are executed according to the real-time operating status. The adaptive control decisions include adaptive package supply decisions and abnormal package identification decisions. Based on the adaptive control decision results, the sorting line is controlled to perform adaptive sorting and processing.

Images