An automatic sorting system based on machine vision

Abstract

This invention relates to the field of image analysis technology, specifically to an automated sorting system based on machine vision. The system includes: a region analysis unit, which constructs a two-dimensional planar map reflecting the distribution of goods for sale based on the shelf locations and product placement within a supermarket, determining the product set corresponding to each shelf; a trajectory analysis unit, which acquires the movement trajectories of each customer and constructs a product interaction set corresponding to each customer based on the two-dimensional planar map and the movement trajectories; a recycling marking unit, which marks the set of the same group of target products as a product return collection, acquires the recycling trajectory corresponding to each product return collection, and determines multiple product interaction sets corresponding to the product return collection based on the intersection of the recycling trajectory and the movement trajectory; and a sorting and recycling unit, which constructs a corresponding comparison image library for the multiple product interaction sets corresponding to the product return collection, and identifies and sorts the target products within the product return collection based on the comparison image library and image recognition technology.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and more specifically to an automated sorting system based on machine vision. Background Technology

[0002] Machine vision-based automated sorting systems are automated systems that use cameras, sensors, and artificial intelligence algorithms to "see" objects and automatically identify and classify them according to preset rules (such as shape, size, color, texture, barcode, text, defects, etc.). They guide robotic arms, pneumatic nozzles, conveyor belt levers, or other actuators to sort objects to different target locations. They are commonly used in scenarios such as logistics parcel sorting, fruit grading, and waste recycling.

[0003] While existing automated sorting systems can handle material sorting tasks in the aforementioned scenarios well, they lack interoperability and are typically only usable in specific situations. In supermarket shopping scenarios, items that customers cancel their purchases need to be re-sorted and placed back on the original shelves, which also requires a significant amount of manual labor. Therefore, an automated sorting system suitable for this scenario is needed to replace manual sorting and improve product replenishment efficiency. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the existing technology, the present invention provides an automatic sorting system based on machine vision, which can effectively solve the problem that the existing automatic sorting system is difficult to apply to the scenario of canceled product sorting in supermarket shopping.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] This invention provides an automated sorting system based on machine vision, comprising at least:

[0007] The regional analysis unit constructs a two-dimensional planar map reflecting the distribution of goods for sale based on the location of shelves and the arrangement of goods on the shelves within the supermarket, and determines the set of goods for each shelf.

[0008] The trajectory analysis unit acquires the movement trajectory of each customer and constructs a product interaction set for each customer based on a two-dimensional planar map and the movement trajectory. The product interaction set contains multiple products that the customer has contacted and purchased.

[0009] The recycling marking unit marks the goods that need to be recycled and sorted as target goods, and marks the set of the same group of target goods as the goods return collection. It obtains the recycling trajectory corresponding to each goods return collection, and determines multiple goods interaction sets corresponding to the goods return collection based on the intersection relationship between the recycling trajectory and the movement trajectory.

[0010] The sorting and recycling unit uses multiple interactive sets of goods to construct a corresponding comparison image library. Based on the comparison image library and image recognition technology, the target goods in the goods collection are identified. The sorting equipment sorts the target goods into the area recycling bins corresponding to different goods classification areas. Each area recycling bin corresponds to multiple goods classification areas on the same route.

[0011] Furthermore, the process of constructing the two-dimensional planar diagram is as follows:

[0012] The floor plan of the supermarket is called the general floor plan. On the general floor plan, the shaded areas corresponding to each product shelf are drawn and called shelf areas. Each shelf area is assigned a shelf number.

[0013] Construct multiple rectangular product category areas. All products for sale within the same product category area belong to the same product category. Divide all shelf areas into multiple different product category areas.

[0014] The outer perimeter of the shelf area is divided into multiple unit intervals, and a corresponding shelf product set is constructed based on the types of goods for sale within each unit interval.

[0015] Furthermore, the process of constructing the shelf product set is as follows:

[0016] The product shelf corresponding to the shelf area is recorded as the target shelf. The front image of the target shelf is collected along the outer perimeter of the shelf area and recorded as the shelf image. All products for sale in the shelf image and the rectangular area corresponding to each product for sale are recorded as the single product area.

[0017] Get the width of all individual product areas and extract the maximum width as the segment width. Use the segment width as the unit width to divide the outer perimeter of the shelf area into multiple unit intervals. Assign an interval number to each unit interval in turn, and bind multiple individual product areas located in the unit interval to the corresponding interval number.

[0018] Construct a shelf product set, in which:

[0019] The shelf product set contains multiple unit subsets, each unit subset corresponds to a unit range, and each unit subset contains multiple products for sale. The products for sale in a unit subset correspond one-to-one with the single product areas in its corresponding unit range.

[0020] Furthermore, the interval segments and interval numbers corresponding to each unit interval are marked on the outer perimeter of the shelf area, and any unit subset is bound to its two adjacent unit subsets to form a range set.

[0021] Furthermore, the process of constructing the product interaction set is as follows:

[0022] The system obtains the complete shopping trajectory of customers in the supermarket and plots the corresponding curve on a two-dimensional plane as the trajectory curve. The system has a preset unit time interval. The system records the customer's position once every unit time interval and marks the corresponding temporary point on the trajectory curve. Each temporary point is assigned a temporary value, which is equal to the number of times the temporary point overlaps.

[0023] The site plan is divided into multiple square unit areas, with the side length of each unit area being a preset value. Based on the distribution of temporary dwell points within each unit area, key areas where customer dwell time exceeds a threshold are selected, and the corresponding temporary dwell points for each key area are determined.

[0024] Based on the location of the temporary mass point, the contact interval on the shelf area is determined. The set of ranges corresponding to the contact intervals is bound to the customer trajectory curve. All range sets bound to the same customer trajectory curve are obtained and their union is calculated to obtain the product interaction set.

[0025] Furthermore, the process for determining the temporary mass point is as follows:

[0026] A two-dimensional coordinate system is constructed within the key area, and the coordinates of each temporary point are marked. Temporary points with a temporary value greater than 1 are selected and recorded as target temporary points. The coordinates of the target temporary points are then obtained. Where i represents the index of the target temporary holding point, i=1,2,…,j, and j represents the total number of target temporary holding points. The temporary holding value of the target temporary holding point is denoted as Substitute into the formula The coordinates of the temporary mass point are obtained by performing calculations. And mark the corresponding locations in the key areas.

[0027] Furthermore, the binding process between the product interaction set and the product feedback set is as follows:

[0028] Each employee in the supermarket is assigned an identification number. The sequence number of each product collection is determined according to the order in which different products are collected and brought to the sorting area. The identification number of the employee who collects the product is then linked to the product collection number.

[0029] The movement trajectory of the staff is identified by the image recognition algorithm and linked to the corresponding goods return collection, which is recorded as the collection trajectory of the goods return collection;

[0030] Based on the recycling trajectory, the recycling area of ​​the product recycling collection is determined, and the trajectory curve that intersects with the recycling area is recorded as the target curve. The product interaction set corresponding to all target curves is obtained and bound to the corresponding product recycling collection.

[0031] Furthermore, the target curve and its associated product collection must satisfy the following conditions:

[0032] Condition 1: The target curve is generated before the commodity return collection is generated;

[0033] Condition 2: The target curve must have at least one bound range set before passing through the recycling area.

[0034] Furthermore, the process of linking the designated recycling bins with the product sorting areas is as follows:

[0035] There are multiple pre-defined placement routes with sorting areas as their endpoints. The product classification areas located on both sides of the placement route are bound to the placement route. The product classification areas bound to the multiple placement routes cover all product classification areas in the supermarket. Each placement route is bound to a regional recycling bin, and the regional recycling bin is bound to all product classification areas corresponding to that placement route. The set of on-sale products corresponding to the regional recycling bin is recorded as the sorting dataset.

[0036] Furthermore, the process of identifying and recycling target goods is as follows:

[0037] S1: Collect the union of multiple product interaction sets corresponding to each product and record it as the total interaction set;

[0038] S2: Individual collection of each item;

[0039] S3: Obtain image data of all products for sale in the interactive master set and build a comparison image library; collect image data of each target product in the product recycling set and compare them one by one with the image data in the comparison image library.

[0040] Based on the comparison results, the corresponding on-sale products are identified and bound to the target product. According to the sorting dataset to which the on-sale products belong, they are sorted into the corresponding area recycling bins.

[0041] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0042] 1. This invention maps the layout of supermarket shelves into a structured two-dimensional plan and combines it with the construction of shelf product sets to achieve refined and spatial product management. It intuitively presents the regional division of different product categories, and at the same time, it can accurately locate the product range corresponding to any shelf location by binding the interval sequence number of the shelf product set, which helps visually identify the product distribution.

[0043] 2. This invention intelligently collects customer movement trajectories and constructs a product interaction set, which can completely record the customer's shopping path and convert it into a trajectory curve with temporary stop point markers. The temporary stop value calculation mechanism accurately reflects the customer's dwell time in each area. Combined with the preset unit area division and temporary stop parameter formula, it automatically identifies key areas and effectively filters out hot spots where customers may interact with products. The system accurately locates the temporary stop mass point by calculating the centroid of the temporary stop point in the key area and associates it with the nearest shelf area. By using the binding relationship between contact anchor points and range sets to construct a product interaction set, it can infer the range of products that the customer actually comes into contact with.

[0044] 3. This invention achieves efficient recycling and accurate classification of scattered goods in supermarkets through intelligent trajectory tracking and sorting logic design. It spatially and temporally links the recycling trajectory of staff with the shopping trajectory of customers, and establishes a complete recycling traceability system by setting the item collection serial number and binding identification employee number. By constructing an interactive set as the benchmark for comparison image library, the system can quickly identify target items and automatically sort them to the corresponding recycling bins. This reduces the error rate of manual sorting and shortens the product restocking cycle, realizing full automation from product discovery and trajectory analysis to intelligent sorting. This not only optimizes supermarket operating costs but also effectively improves inventory turnover efficiency and customer shopping experience by reducing the time that products stay in non-designated areas. Attached Figure Description

[0045] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0046] Figure 1 This is an overall module block diagram of the present invention. Detailed Implementation

[0047] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] The present invention will be further described below with reference to embodiments.

[0049] See Figure 1An automated sorting system based on machine vision, suitable for sorting and organizing goods in supermarkets, records goods currently on sale as goods for sale, including at least:

[0050] The regional analysis unit constructs a two-dimensional plan view reflecting the distribution of goods for sale based on the shelf locations and product placement within the supermarket, where:

[0051] Obtain a floor plan of the supermarket, designated as the master plan. On this master plan, draw shaded areas corresponding to each product shelf, designated as shelf areas, and assign a shelf number to each shelf area. Based on the types of goods displayed on different shelves, divide all shelf areas into multiple distinct product category areas. Each product category area is rectangular, and all goods within the same category belong to the same major product category. Construct a shelf product set based on the arrangement characteristics of the goods within each shelf area. This shelf product set allows for the rapid determination of the range of goods available for sale within each section of that shelf area.

[0052] It should be noted that merchandise shelves refer to all equipment and facilities used for displaying goods in supermarkets and stores, such as shelves and display cases. Furthermore, classifying and selling target products is a common practice in existing technology (e.g., dividing goods for sale into daily necessities, fast food, and beverage sections), which will not be elaborated upon further here.

[0053] More specifically, the process of building a shelf product set is as follows:

[0054] The product shelf corresponding to the shelf area is recorded as the target shelf. The front image of the target shelf is collected along the outer perimeter of the shelf area and recorded as the shelf image. Multiple shelf images constitute the complete product display image of the target shelf (shelf images taken from different positions and angles can be combined to form the complete product display image of the target shelf). Based on the image recognition algorithm, all products for sale in the shelf image and the rectangular area corresponding to each product for sale are identified and recorded as the single product area. There is only one product for sale in the single product area (a single shelf image is composed of multiple single product areas).

[0055] Get the width of all single-item areas (in all shelf images corresponding to the same shelf area) and extract the maximum width as the segment width. Divide the outer perimeter of the shelf area into multiple unit intervals using the segment width as the unit width. Assign an interval number to each unit interval in turn. Bind multiple single-item areas (including some single-item areas located in the unit interval) located in the unit interval to the corresponding interval number. Mark the interval line segments and interval numbers corresponding to each unit interval on the outer perimeter of the shelf area.

[0056] In another embodiment, the individual product area can be manually divided based on product labels, with each product label corresponding to a rectangular area, which is an individual product area.

[0057] A shelf product set is constructed based on the interval numbers corresponding to all products for sale within the shelf area. The shelf product set contains multiple unit subsets, each unit subset corresponds to an interval number, and each unit subset contains multiple products for sale. Any unit subset (A) is bound to two adjacent unit subsets (B and C) (the unit intervals are spatially adjacent) to form a range set (the range set is bound to the interval number of unit subset A). In other words, the approximate types of products for sale within a certain range around the shelf area can be determined based on the range set, thereby narrowing down the scope of product type investigation.

[0058] It should be noted that constructing a range set enables the range set to include all available products within the unit subset A and all available products in its surrounding area, thereby increasing the coverage of available product types. When a customer is located within the unit interval corresponding to the unit subset A, the available products they may take are included in the range set.

[0059] Compared to ordinary product sets, shelf product sets have geometric characteristics: the elements in the set correspond to different ranges around the shelf area. This allows multiple products for sale to be identified based on any point on the periphery of the shelf area. Notably, different shelf areas are distinguished by different serial numbers, ensuring that each subset has a unique serial coordinate system containing both the shelf area number and the range number.

[0060] The trajectory analysis unit acquires the movement trajectory of each customer and constructs a product interaction set for each customer based on the movement trajectory. The product interaction set contains all the products for sale that the customer may have come into contact with or purchased. Specifically:

[0061] When a customer enters the supermarket, images of the customer's appearance (including but not limited to facial images and full-body images, which can quickly distinguish the customer's identity) are collected, and each customer is assigned an entry number based on the appearance image. Based on multiple image acquisition devices evenly distributed in the supermarket, the complete shopping trajectory of the customer is obtained and the corresponding curve is drawn on the overall plan and recorded as the trajectory curve. A unit time is preset (1 second in a specific embodiment), and the value of the unit time ranges from 1 second to 5 seconds. The customer's position is recorded once every unit time interval, and the corresponding temporary point is marked on the trajectory curve. Each temporary point is assigned a temporary value, which is equal to the number of times the temporary point overlaps, that is, the number of times the same point is marked as a temporary point.

[0062] It is worth noting that identity recognition based on appearance images, such as facial images, is an existing technology and will not be elaborated on here. When collecting customer appearance image data, edge processing is adopted, that is, after the facial features are extracted on the local device, the original appearance image is deleted immediately, and only the feature code (not the original appearance image) is uploaded to ensure that the data is not easy to steal and avoid customer privacy leakage.

[0063] The site plan is divided into multiple square unit areas, with a preset side length (1m in one specific embodiment). The side length of the unit area ranges from 1m to v*t, where t represents the unit time and v represents the human walking speed. The sum of the dwell times of all dwell points within each unit area is calculated and substituted into the formula. The calculation is performed to obtain the temporary parameter R, where t represents the unit time. This represents the sum of the temporarily retained values. The dwell time parameter reflects the length of time a customer spends within a given unit area (with units consistent across units). A higher dwell time parameter indicates a longer customer stay within that unit area. A dwell time threshold is preset; all units with dwell time parameters exceeding this threshold are designated as critical areas. If a customer's dwell time within a critical area exceeds the preset threshold, it indicates a high probability that the customer will interact with products on nearby shelves within that unit area. The dwell time threshold... The calculation formula is , where v represents human walking speed (approximately 1.5 m / s).

[0064] Based on the distribution of temporary points within the key area, the temporary mass points of the key area are determined. The temporary mass points are located at the centroids of multiple temporary mass points. The shelf area closest to the temporary mass point is recorded as the contact shelf corresponding to that temporary mass point (each temporary mass point corresponds to one contact shelf). The shortest line segment perpendicular to the outer perimeter of the contact shelf is drawn through the temporary mass point, and the perpendicular point is recorded as the contact anchor point. The unit interval where the contact anchor point is located is recorded as the contact interval of that contact anchor point. The set of ranges corresponding to the contact interval is bound to the customer trajectory curve. All sets of ranges bound to the same customer trajectory curve are obtained and their union is calculated to obtain the product interaction set.

[0065] It should be noted that the product interaction set refers to the set of on-sale products that customers may interact with throughout the shopping process. Interaction refers to a series of actions and behaviors with the purchase as the main focus, including picking up and observing, putting into the shopping cart, or putting back on the shelf. This set only represents the set of products that may interact with the customer and does not represent the set of products that the customer will ultimately purchase. However, it is certain that the set of products that the customer will ultimately select will be a subset of the product interaction set.

[0066] More specifically, the process for determining the temporary mass point is as follows:

[0067] A two-dimensional coordinate system is constructed within the key area, and the coordinates of each temporary point are marked. Temporary points with a temporary value greater than 1 are selected and recorded as target temporary points. The coordinates of the target temporary points are then obtained. Where i represents the index of the target temporary holding point, i=1,2,…,j, and j represents the total number of target temporary holding points. The temporary holding value of the target temporary holding point is denoted as (The corresponding target temporary retention point with index i), substitute into the formula The coordinates of the temporary mass point are obtained by performing calculations. And mark the corresponding locations in the key areas.

[0068] The recycling marking unit marks the goods that need to be recycled and sorted as target goods, and the set of the same group of target goods as the goods return collection.

[0069] Target products refer to those items that customers casually place or leave in supermarkets that are not on their original shelves. For example, if a customer puts some items in their shopping cart but then abandons the purchase during the shopping trip and leaves these items in other locations in the supermarket, these items need to be collected and sorted before they can be placed back on the shelves. These items are considered target products. Multiple target products that are left in the same location in the same batch are recorded as product recollection (for example, target products in the same shopping bag, shopping basket, or shopping cart are recorded as the same batch).

[0070] Specifically, each employee in the supermarket is assigned an identification number. The sequence number of each product collection is determined based on the order in which different product collections are returned to the sorting area. This sequence number is used to distinguish different product collections. The identification number of the employee who collected the product collection is then linked to the collection number (meaning that each product collection can be traced back to its source, identifying the employee who collected it). The movement trajectory of the employee is identified using an image recognition algorithm and linked to the corresponding product collection, which is recorded as the collection trajectory of that product collection. The collection area of ​​the product collection is determined based on the collection trajectory. The trajectory curve that intersects with the collection area is recorded as the target curve. The product interaction set corresponding to the target curve is then linked to that product collection (the same product collection corresponds to one or more product interaction sets).

[0071] The target curve and its associated product return collection must meet the following conditions:

[0072] Condition 1: The target curve is generated before the commodity collection is generated (i.e., the commodity collection sequence number is determined at the time of generation).

[0073] Condition 2: The target curve must have at least one bound range set before passing through the recycling area (i.e., the customer must have had at least one product interaction behavior before passing through the recycling area for the target curve to be considered related to the corresponding product recycling collection).

[0074] By applying the above conditions, the range of the target curve can be further narrowed, reducing interference from target curves unrelated to commodity recycling.

[0075] More specifically, the process for determining the recovery trajectory is as follows:

[0076] The system obtains the label time of the returned goods (i.e. the time when they are returned to the sorting area). There is a preset interception time. The time interval with the label time as the end point and the interception length as the interception time is called the interception interval. The movement trajectory of the staff bound to the returned goods is called the interception trajectory. The interception trajectory located within the interception interval is called the recovery trajectory.

[0077] More specifically, the process for determining the recycling area is as follows:

[0078] The system acquires the recovery trajectory and identifies the temporary points and corresponding temporary values ​​on the trajectory. It calculates the sum of the temporary values ​​of the recovery trajectory in each unit area. A detection time threshold is preset. When the sum of the temporary values ​​of the recovery trajectory in any unit area is greater than the detection time threshold, the unit area is recorded as the recovery area and bound to the recovery trajectory.

[0079] To ensure accurate identification of recycling areas, staff can be required to remain at the location where the goods were found for a fixed period of time during the recycling process. This fixed period must exceed a preset threshold for the time spent at the location. Furthermore, the recycling work must be carried out by dedicated staff.

[0080] It should be noted that the recycling area refers to the unit area where staff stay for a relatively long time when moving within the supermarket. It usually corresponds to the area where staff are collecting goods. The same recycling route may be associated with multiple recycling areas because staff may collect multiple goods at the same time during a recycling process. Therefore, using multiple recycling areas can cover the recycling locations of all goods collection points, so as to limit the range of target goods within the collection points.

[0081] The sorting and recycling unit uses image recognition technology to identify the target products in the recycling bin. The sorting equipment sorts the target products into the recycling bins corresponding to different product categories. Each recycling bin corresponds to multiple product categories on the same route. This allows staff to rearrange the target products in the recycling bins by simply walking along one route, maximizing route efficiency and reducing sorting time.

[0082] Specifically, the process of linking the designated recycling bins with the product sorting areas is as follows:

[0083] There are multiple pre-defined placement routes with sorting areas as their endpoints. The product classification areas located on both sides of the placement route are bound to the placement route. The product classification areas bound to the multiple placement routes cover all product classification areas in the supermarket. Each placement route is bound to a regional recycling bin, which is bound to all product classification areas corresponding to that placement route. That is, the target product in the regional recycling bin belongs to the on-sale products in any of the bound product classification areas. The set of on-sale products corresponding to the regional recycling bin (the union of the sets of on-sale products in all the product classification areas bound to it) is denoted as the sorting dataset.

[0084] It should be noted that the route planning algorithm can be used to construct the placement route required in this embodiment, or it can be constructed manually by staff. The route planning algorithm is existing technology and will not be described in detail here.

[0085] The process of identifying the target product is as follows:

[0086] S1: Obtain the union of multiple product interaction sets corresponding to each product return collection, denoted as the total interaction set. The product return collection is a subset of the total interaction set. In other words, the total interaction set must contain all target products in the product return collection.

[0087] S2: Each product collection point is collected separately, meaning different product collection points are sorted separately to avoid sorting interference between different product collection points;

[0088] S3: Obtain image data of all products for sale in the overall interactive set and build a comparison image library. Collect image data of each target product in the product recycling set and compare them one by one with the image data in the comparison image library. Based on the comparison results, determine the products for sale corresponding to the target products and bind them to the target products. According to the sorting dataset to which the products for sale belong, sort them into the corresponding recycling bins.

[0089] It is worth noting that although the interoperability between automated sorting systems in different scenarios is not good, the sorting equipment can be applied. In a specific embodiment, the target product sorting equipment mentioned in this invention is composed of a combination of a conveyor belt and a mechanical push rod / lever. The target product is recycled by pushing it laterally out of the conveyor belt and into the area recycling bin.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automated sorting system based on machine vision, characterized in that, include: The regional analysis unit constructs a two-dimensional planar map reflecting the distribution of goods for sale based on the location of shelves and the arrangement of goods on the shelves within the supermarket, and determines the set of goods for each shelf. The trajectory analysis unit acquires the movement trajectory of each customer and constructs a product interaction set for each customer based on a two-dimensional planar map and the movement trajectory. The product interaction set contains multiple products that the customer has contacted and purchased. The recycling marking unit marks the goods that need to be recycled and sorted as target goods, and marks the set of the same group of target goods as the goods return collection. It obtains the recycling trajectory corresponding to each goods return collection, and determines multiple goods interaction sets corresponding to the goods return collection based on the intersection relationship between the recycling trajectory and the movement trajectory. The sorting and recycling unit uses multiple interactive sets of goods to construct a corresponding comparison image library. Based on the comparison image library and image recognition technology, the target goods in the goods collection are identified. The sorting equipment sorts the target goods into the area recycling bins corresponding to different goods classification areas. Each area recycling bin corresponds to multiple goods classification areas on the same route.

2. The automated sorting system based on machine vision according to claim 1, characterized in that, The process of constructing a two-dimensional planar diagram is as follows: The floor plan of the supermarket is called the general floor plan. On the general floor plan, the shaded areas corresponding to each product shelf are drawn and called shelf areas. Each shelf area is assigned a shelf number. Construct multiple rectangular product category areas. All products for sale within the same product category area belong to the same product category. Divide all shelf areas into multiple different product category areas. The outer perimeter of the shelf area is divided into multiple unit intervals, and a corresponding shelf product set is constructed based on the types of goods for sale within each unit interval.

3. The automated sorting system based on machine vision according to claim 2, characterized in that, The process of building a shelf product set is as follows: The product shelf corresponding to the shelf area is recorded as the target shelf. The front image of the target shelf is collected along the outer perimeter of the shelf area and recorded as the shelf image. All products for sale in the shelf image and the rectangular area corresponding to each product for sale are recorded as the single product area. Get the width of all individual product areas and extract the maximum width as the segment width. Use the segment width as the unit width to divide the outer perimeter of the shelf area into multiple unit intervals. Assign an interval number to each unit interval in turn, and bind multiple individual product areas located in the unit interval to the corresponding interval number. Construct a shelf product set, in which: The shelf product set contains multiple unit subsets, each unit subset corresponds to a unit range, and each unit subset contains multiple products for sale. The products for sale in a unit subset correspond one-to-one with the single product areas in its corresponding unit range.

4. The automated sorting system based on machine vision according to claim 3, characterized in that, Mark the interval segments and interval numbers corresponding to each unit interval on the outer perimeter of the shelf area, and bind any unit subset with its two adjacent unit subsets to form a range set.

5. The automated sorting system based on machine vision according to claim 4, characterized in that, The process of building a product interaction set is as follows: The system obtains the complete shopping trajectory of customers in the supermarket and plots the corresponding curve on a two-dimensional plane as the trajectory curve. The system has a preset unit time interval. The system records the customer's position once every unit time interval and marks the corresponding temporary point on the trajectory curve. Each temporary point is assigned a temporary value, which is equal to the number of times the temporary point overlaps. The site plan is divided into multiple square unit areas, with the side length of each unit area being a preset value. Based on the distribution of temporary dwell points within each unit area, key areas where customer dwell time exceeds a threshold are selected, and the corresponding temporary dwell points for each key area are determined. Based on the location of the temporary mass point, the contact interval on the shelf area is determined. The set of ranges corresponding to the contact intervals is bound to the customer trajectory curve. All range sets bound to the same customer trajectory curve are obtained and their union is calculated to obtain the product interaction set.

6. The automated sorting system based on machine vision according to claim 5, characterized in that, The process for determining the temporary mass point is as follows: A two-dimensional coordinate system is constructed within the key area, and the coordinates of each temporary point are marked. Temporary points with a temporary value greater than 1 are selected and recorded as target temporary points. The coordinates of the target temporary points are then obtained. Where i represents the index of the target temporary holding point, i=1,2,…,j, and j represents the total number of target temporary holding points. The temporary holding value of the target temporary holding point is denoted as Substitute into the formula The coordinates of the temporary mass point are obtained by performing calculations. And mark the corresponding locations in the key areas.

7. The automated sorting system based on machine vision according to claim 1, characterized in that, The binding process between the product interaction set and the product feedback set is as follows: Each employee in the supermarket is assigned an identification number. The sequence number of each product collection is determined according to the order in which different products are collected and brought to the sorting area. The identification number of the employee who collects the product is then linked to the product collection number. The movement trajectory of the staff is identified by the image recognition algorithm and linked to the corresponding goods return collection, which is recorded as the collection trajectory of the goods return collection; Based on the recycling trajectory, the recycling area of ​​the product recycling collection is determined, and the trajectory curve that intersects with the recycling area is recorded as the target curve. The product interaction set corresponding to all target curves is obtained and bound to the corresponding product recycling collection.

8. The automated sorting system based on machine vision according to claim 7, characterized in that, The target curve and its associated product return collection must meet the following conditions: Condition 1: The target curve is generated before the commodity return collection is generated; Condition 2: The target curve must have at least one bound range set before passing through the recycling area.

9. The automated sorting system based on machine vision according to claim 1, characterized in that, The process of linking designated recycling bins with product sorting areas is as follows: There are multiple pre-defined placement routes with sorting areas as their endpoints. The product classification areas located on both sides of the placement route are bound to the placement route. The product classification areas bound to the multiple placement routes cover all product classification areas in the supermarket. Each placement route is bound to a regional recycling bin, and the regional recycling bin is bound to all product classification areas corresponding to that placement route. The set of on-sale products corresponding to the regional recycling bin is recorded as the sorting dataset.

10. An automated sorting system based on machine vision according to claim 9, characterized in that, The specific process for identifying and recycling target goods is as follows: S1: Collect the union of multiple product interaction sets corresponding to each product and record it as the total interaction set; S2: Individual collection of each item; S3: Obtain image data of all products for sale in the interactive master set and build a comparison image library; collect image data of each target product in the product recycling set and compare them one by one with the image data in the comparison image library. Based on the comparison results, the corresponding on-sale products are identified and bound to the target product. According to the sorting dataset to which the on-sale products belong, they are sorted into the corresponding area recycling bins.

Images