Method and system for improving sorting accuracy based on AI algorithm sorting line

Abstract

The invention discloses a method and system for improving sorting accuracy based on an AI algorithm sorting line, and relates to the technical field of image sample collection, and the method comprises the steps: obtaining a first original picture of a transported object through a sorting equipment end after the transported object used for target sample collection is determined; the transported object falling on the backflow device is sent to the conveying belt again to be shot, and a second original picture is obtained; and inputting the first or second original picture into the first detection model and the second detection model respectively, and if it is detected that the identification results of the two models of the first or second original picture are inconsistent, expanding the target sample set by using the original picture. By implementing the method, the sample image of the target category of the real production line can be quickly obtained with high quality and low cost, so that the intelligent recognition model carried by the sorting equipment is trained, and the sorting accuracy of the sorting equipment is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image sample collection, and in particular to a method and system for improving sorting accuracy based on an AI algorithm sorting line. Background Art

[0002] In the fields of grain, food, minerals, chemicals and other industries, it is generally necessary to sort items. In order to improve work efficiency, sorting equipment such as optical sorters are generally selected for automatic sorting. They are mainly equipped with intelligent recognition models to accurately identify different types of targets on the conveyor belt, such as plastics, metals, glass, etc., to achieve fast and high-purity sorting. In this way, the quality of the data sample has a greater impact on the recognition accuracy of the sorting equipment. The more comprehensive and accurate the data sample, the higher the accuracy of the sorting equipment when it is identified through the intelligent recognition model.

[0003] The intelligent recognition model carried by the sorting equipment often needs to be trained with a large number of sample images to improve the accuracy of intelligent recognition. In related technologies, the collection of sample images mainly relies on manually building a camera platform to take pictures of the collected target category objects, thereby obtaining target sample images and constructing a training sample set.

[0004] However, the method of obtaining target sample images in related technologies mainly relies on manual layout shooting, which not only consumes a lot of human resources but also has low sample collection efficiency. Summary of the invention

[0005] The present application provides a method and system for improving the sorting accuracy of a sorting line based on an AI algorithm, which is used to quickly, high-quality and low-cost obtain sample images of target categories of a real production line, and then train the intelligent recognition model carried by the sorting equipment to improve the sorting accuracy of the sorting equipment.

[0006] In a first aspect, the present application provides a method for improving sorting accuracy based on an AI algorithm sorting line, which is applied to a sample collection system, and the method comprises: determining a transport object for target sample collection, the transport object being placed on a conveyor belt; Acquire a first original image of the transport object through a sorting device end, where a camera for photographing is installed on the sorting device end; The transport object dropped on the reflow device is put back onto the conveyor belt for shooting to obtain a second original picture. There is a height difference between the reflow device and the end of the conveyor belt, which is used to change the contact position between the transport object and the conveyor belt. Inputting the original image into the first detection model and the second detection model respectively, and judging whether the recognition results of the two models are consistent, the original image includes the first original image and the second original image, the first detection model is installed on the sorting device end, and the second detection model is installed on the cloud server; If it is detected that the recognition results of the two models of the original image are inconsistent, the original image is used to expand the target sample set.

[0007] Through the above-mentioned embodiments, the sample collection system obtains the image information of the transported object on the conveyor belt through the sorting device end, and then changes the shooting angle of the transported object on the conveyor belt through the reflux device, thereby realizing multi-angle shooting of the transported object. At the same time, for the original pictures taken, by comparing the recognition results of the first detection model and the second detection model, valuable sample original pictures are screened out to expand the target sample set, thereby improving the comprehensiveness and accuracy of target sample collection. In addition, through the cooperation of the sorting device end and the reflux device, the process of automatically collecting target samples is realized, which reduces manual participation and significantly improves the efficiency and quality of sample collection.

[0008] In some embodiments, the steps of inputting the original image into the first detection model and the second detection model respectively and determining whether the recognition results of the two models are consistent specifically include: Inputting the original image into the first detection model to identify the first category information and coordinate information of each transport object in the original image; Inputting the original image into the second detection model to identify the second category information of the transport object corresponding to each coordinate information; If it is detected that the first category information is the same as the second category information, then it is determined that the recognition results of the two models are consistent; If it is detected that the first category information is different from the second category information, it is determined that the two model recognition results are inconsistent.

[0009] Through the above embodiment, the sample collection system performs a second verification on the identification type of each transport object identified by the first detection model through the second detection model, thereby improving the accuracy of category recognition of the original image, thereby improving the reliability of sample screening, ensuring that the images expanded to the target sample set have higher value, and helping to improve the recognition capability of the entire system.

[0010] In some embodiments, the step of inputting the original image into the second detection model and identifying the second category information of the transport object corresponding to each coordinate information specifically includes: A target image corresponding to each transport object is captured from the original image, and the position of the target image is determined according to the coordinate information; Convert the target image into a three-channel image of a preset format; The three-channel image is input into the second detection model to identify the second category information corresponding to the three-channel image.

[0011] Through the above embodiment, the sample collection system captures an image of a single transport object from the original image as the target image, converts it into a three-channel image in a preset format, and then inputs it into the second detection model for recognition. Among them, by preprocessing the target image, the influence of background noise can be reduced, so that the second detection model can be more focused on the characteristics of the target object, thereby improving the recognition accuracy and facilitating the verification of the recognition result of the first model.

[0012] In some embodiments, after the step of determining a transport object for target sample collection, the method further includes: Match the height difference corresponding to the transport object from the database; Adjust the height difference between the conveyor belt and the return device according to the height difference value.

[0013] Through the above embodiments, the sample collection system introduces a mechanism for dynamically adjusting the height difference between the conveyor belt and the reflux device. By matching the height difference value corresponding to the transported object from the database to adjust the height, it is possible to handle objects of various sizes and shapes, so that the landing point of the transported object can be changed during the process of falling from the conveyor belt to the reflux device, so that when the reflux device re-enters the conveyor belt for shooting, original pictures at different angles can be obtained, thereby improving the quality of sample collection.

[0014] In some embodiments, if it is detected that the recognition results of the two models of the original image are inconsistent, the step of using the original image to expand the target sample set specifically includes: Call the SAM segmentation algorithm to automatically annotate the original image and generate annotation data; Receive adjustment parameters of the annotation data from technicians; Fine-tune the annotation data according to the adjustment parameters to obtain valid annotation data of the original image; Use valid labeled data to expand the target sample set.

[0015] Through the above embodiments, the sample collection system automatically annotates the original image by introducing the SAM segmentation algorithm, and generates high-quality effective annotation data in combination with the adjustment of the technician. This human-machine combination not only improves the efficiency of annotation, but also ensures the accuracy of annotation. Automatic annotation reduces the manual workload, while manual adjustment makes up for the defects that may exist in the automatic algorithm. This method can not only quickly expand the high-quality sample set, but also continuously improve the recognition ability of the system, providing strong support for the iterative optimization of the model.

[0016] In some embodiments, after the step of using the original image and the valid annotation data to expand the target sample set, the method further includes: Receive the data increment threshold set by the user; After detecting that the amount of valid annotated data in the target sample set exceeds the data increment threshold, the newly added valid annotated data is input into the first detection model and the second detection model for tuning training.

[0017] Through the above embodiments, the sample collection system can timely utilize the newly added valid annotated data to improve the model performance by setting a threshold and performing model tuning training after the threshold is reached. This mechanism ensures that the model can continuously learn new sample features and adapt to the ever-changing environment.

[0018] In some embodiments, after the step of sending the transport object dropped on the reflow device back onto the conveyor belt for photographing to obtain a second original picture, the method further includes: Count the number of return flows of transported objects; When it is detected that the number of reflows exceeds a preset cycle threshold, the transported object dropped on the reflow device will no longer be sent back onto the conveyor belt for photographing.

[0019] Through the above embodiments, the sample collection system avoids the waste of resources and system inefficiency caused by repeated circulation of the same object by limiting the number of refluxes of the transported object. This mechanism not only optimizes the operating efficiency of the system, but also avoids damage to the object that may be caused by excessive reflux. At the same time, this design also increases the diversity of samples and prevents the system from focusing too much on certain specific samples.

[0020] In a second aspect, the present application provides a sample collection system, the sample collection system comprising: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions. The one or more processors call the computer instructions so that the sample collection system can implement a method for improving sorting accuracy based on an AI algorithm sorting line provided in the above embodiment, which will not be repeated here.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on a sample collection system, the sample collection system can implement a method for improving sorting accuracy based on an AI algorithm sorting line provided in the above-mentioned embodiment, which will not be described in detail here.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a sample collection system, the sample collection system can implement a method for improving sorting accuracy based on an AI algorithm sorting line provided in the above-mentioned embodiment, which will not be repeated here.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The sample collection system obtains the image information of the transported objects on the conveyor belt through the sorting equipment end, and then changes the shooting angle of the transported objects on the conveyor belt through the reflux device, thereby realizing multi-angle shooting of the transported objects.

[0024] 2. The sample collection system realizes the process of automatically collecting target samples through the cooperation of the sorting equipment end and the reflux device, reduces manual participation, and significantly improves the efficiency and quality of sample collection.

[0025] 3. The sample collection system compares the recognition results of the first detection model and the second detection model for the original pictures taken, screens out valuable sample original pictures to expand the target sample set and uses them to tune and train the model, thereby improving the comprehensiveness and accuracy of target sample collection and the accuracy of model recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a method for improving sorting accuracy based on an AI algorithm sorting line in an embodiment of the present application; Figure 2 This is another flow chart of a method for improving sorting accuracy based on an AI algorithm sorting line in an embodiment of the present application; Figure 3 is a schematic diagram of an exemplary scenario in which a reflow device in an embodiment of the present application re-feeds a transport object dropped from a conveyor belt back into the conveyor belt; Figure 4 is a schematic diagram of an exemplary scenario in which the first detection model mounted on the sorting device in the embodiment of the present application recognizes the original image; Figure 5 It is a schematic diagram of a physical device structure of a sample collection system in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, the application scenarios of the relevant technical embodiments are introduced below.

[0030] In a large city's garbage treatment center, thousands of tons of mixed garbage need to be processed every day. In order to achieve effective recycling of resources, the garbage treatment center introduced an advanced optical sorting machine system to automate the garbage sorting process. This system needs to classify garbage into four categories: recyclables (such as plastic bottles and metal cans), kitchen waste, hazardous waste and other garbage. However, the management of the garbage treatment center soon discovered that although the optical sorting machine greatly improved the sorting efficiency, its accuracy still needs to be improved. In particular, when identifying items with similar shapes but different materials (such as plastic packaging and aluminum foil packaging), and when dealing with contaminated or damaged recyclables, misjudgments often occur. This not only affects the purity of the recyclables, but also increases the workload and cost of subsequent manual corrections. In order to improve the recognition accuracy of the optical sorting machine, the garbage treatment center urgently needs a large number of high-quality and diverse garbage sample images to train and optimize the AI ​​model for garbage classification and recognition. However, the traditional method of manually collecting sample images faces huge challenges: there are many types of garbage, different shapes, and there are health and safety risks. How to efficiently and safely obtain a large number of real garbage sample images has become an urgent problem to be solved.

[0031] To solve the sample collection problem, the waste treatment center initially adopted a common method. They set up a special sample collection station next to the processing line and equipped it with several trained staff members. These staff members randomly selected various types of garbage samples from the garbage processing line every day, and after simple cleaning, they placed them on a special shooting table and used high-definition cameras to take pictures of the garbage from different angles. These pictures were then used to train and update the AI ​​model of the optical sorter.

[0032] However, the method of obtaining target sample images in related technologies mainly relies on manual layout shooting, and the manually arranged shooting environment is different from the actual situation on the production line. The collected images cannot truly reflect the various possible states of the classified garbage on the conveyor belt. This makes the sample collection process not only consume a lot of human resources, but also has low sample collection efficiency and low sample quality.

[0033] In response to the above problems, the present application provides a method and system for improving sorting accuracy based on an AI algorithm sorting line. Based on this, when various types of garbage move on the conveyor belt, the sample collection system can control the high-speed camera installed on the optical sorter to automatically take the first batch of original pictures. And the transported objects after shooting are introduced into a specially designed reflux device, which is used to change the contact position between the garbage and the conveyor belt, so that when the garbage re-enters the conveyor belt, the second batch of original pictures can be taken to capture sample images of different angles and states. On this basis, a dual-model verification mechanism is adopted to simultaneously input these original pictures into the fast response model installed locally on the optical sorter and the high-precision model in the cloud. When the recognition results of the two models are inconsistent, these valuable sample pictures are automatically collected to expand the target sample set. It is achieved to quickly, high-quality and low-cost obtain sample images of target categories of real production lines.

[0034] For ease of understanding, the following describes the process of the method provided by this implementation in combination with the above scenario. Figure 1 The figure is a flow chart of a method for improving sorting accuracy of a sorting line based on an AI algorithm in an embodiment of the present application.

[0035] S101. Determine a transport object for collecting target samples.

[0036] In the process of collecting samples using a sorting device to sort garbage, the user needs to first set up a transport object for collecting target samples on the sample collection system, and the transport object is transported on a conveyor belt. For example, in a specific embodiment, if the sorting device used is an optical sorter, the optical sorter is required to separate plastic garbage and glass garbage. Therefore, the transport object placed on the conveyor belt that needs to be sorted is mixed garbage containing plastic products and glass products. While the mixed garbage is being transported on the conveyor belt, the camera mounted on the optical sorter takes pictures, and the pictures taken are annotated and used for training the detection model to improve the accuracy of the detection model in identifying the type of garbage. Therefore, in this embodiment, the transport object used for collecting target samples is mixed garbage.

[0037] It should be noted that in the fields of grain, food, minerals, chemicals and other industries, it is generally necessary to sort items. In order to improve work efficiency, an optical sorter is generally selected for automatic sorting, mainly by carrying an intelligent recognition model to accurately identify different types of targets on the conveyor belt, such as plastic, metal, glass, etc., to achieve fast and high-purity sorting. For ease of understanding, the embodiments of this application are mainly explained by taking the automatic sorting of garbage as an example. Of course, the technical solution provided by this application is also applicable to sorting work in other fields, which is not limited here.

[0038] It should also be noted that the sorting equipment provided in this application is mainly used for equipment that relies on the image recognition function of an intelligent model to perform sorting work. In actual application scenarios, the sorting equipment includes but is not limited to an optical sorter, which is not limited here.

[0039] S102: Obtain a first original image of the transport object through the sorting device.

[0040] Specifically, the transport objects to be classified generally enter the conveyor belt through the feed port. During the transportation on the conveyor belt, the transport objects to be classified will pass through the sorting equipment end. At this time, the camera installed inside the sorting equipment end will automatically shoot the transport objects on the conveyor belt, obtain the first original picture and save it, providing material for subsequent model training.

[0041] S103, sending the transport object dropped on the reflow device back onto the conveyor belt for photographing to obtain a second original picture.

[0042] After being photographed by the sorting device, the transported object on the conveyor belt falls into the reflux device at the end of the conveyor belt, and then returns to the conveyor belt and passes through the sorting device again to be photographed, obtaining a second original picture. Among them, there is a certain height difference between the end of the conveyor belt and the reflux device, which facilitates the transported object to flip during the falling process, so that the transported object that passes through the reflux device and then returns to the conveyor belt changes its contact position with the conveyor belt. For example, before falling, a plastic bottle is oriented in the positive direction of the conveyor belt, and after falling and returning to the conveyor belt, the plastic bottle is oriented in the reverse direction of the conveyor belt.

[0043] Specific as Figure 3 The figure is a schematic diagram of an exemplary scenario in which the reflow device in the embodiment of the present application sends the transported objects dropped from the conveyor belt back to the conveyor belt. Among them, the conveyor belt A is used to transport the transported objects to the camera of the optical sorting machine device for shooting, and the ramp B and the conveyor belt C together form the reflow device. A camera is installed above the conveyor belt A (the camera is installed on the optical sorting machine device end, and the optical sorting machine device end is not in the Figure 3 ), used to obtain the original image of the object being transported on the conveyor belt, and the direction of the arrow is the direction of movement of the conveyor belt. Figure 4 The dotted line in is a virtual path for transporting the object, which is not completely the same as the actual motion trajectory of the transported object and is not limited here.

[0044] The ramp B is used to receive the transported object dropped from the conveyor belt A, so that the transported object slides onto the conveyor belt C. The conveyor belt C then re-transmits the transported object that slides down the ramp B onto the conveyor belt A, so that the transported object dropped from the conveyor belt A enters the conveyor belt A again for shooting, and obtains the second original picture.

[0045] It should be noted that due to the height difference between the end of the conveyor belt and the slope B, the transported object will flip to a certain extent during the falling process. In addition, the transported object may be deflected due to collision during the process of entering the conveyor belt C from the slope B, causing the transported object to be transported to C and reach the conveyor belt A. There are differences in the direction, position and arrangement order compared to the last time on the conveyor belt A, resulting in a certain difference in shooting angle between the second original picture and the first original picture.

[0046] S104: Input the original image into the first detection model and the second detection model respectively, and determine whether the recognition results of the two models are consistent.

[0047] After obtaining a wealth of original images, the sample collection system needs to screen out truly valuable samples to optimize the performance of the entire system. This step introduces two detection models to judge the quality and value of each image by comparing their recognition results.

[0048] The sample collection system inputs each original image (including the first original image and the second original image) into two detection models, where the first detection model is installed on the sorting device and the second detection model runs on the cloud server. The two models have similar functions, and the second detection model is superior to the first detection model in terms of resource allocation and model complexity.

[0049] For ease of understanding, the construction process of the first detection model and the second detection model is introduced below. Taking the first detection model as an example, the user collects sample images and annotates them (for example, the transparent green bottle in the sample image is annotated as green_bottle, the transparent blue bottle is annotated as blue_bottle, etc.), and obtains a data set. Then generate a training set in yolo format based on the data set. Next, use the yolov8 obb target detection algorithm and the above training set for single target detection training to identify the target objects in the sample image and the relative position of the target objects. Then classify the target objects in the above data set, and according to the coordinate information and category information in the annotation file, extract the target image (the image corresponding to the target object) from the original image (sample image) and place it in the category folder corresponding to the target annotation category. Data processing is performed on the images in the above multiple category folders, and the sizes of the target images are converted to the same size (such as 112*112, 224*224, 320*320). Finally, the transformer classification model is used to train the data in the above folders so that the classification model generated by the training has the ability to correctly classify the target, thereby obtaining the first detection model.

[0050] Based on the above content, it can be known that the working process of the first detection model is: after detecting the original image input, each target object appearing in the original image is identified through the internal YOLOv8 OBB single target detection, and multiple target images and their corresponding position information are obtained, and then each target image is input into the transformer classification model for calculation to obtain the category information corresponding to each target image.

[0051] It should be noted that the Transformer classification model provided in the embodiment of the present application is a deep learning model based on the self-attention mechanism, which is widely used in the fields of text classification, image classification, etc. No further description will be given here.

[0052] In addition, the training process of the second detection model is basically the same as that of the first detection model. The difference is that the second detection model is installed on a cloud server with better computing resources and a higher level of classification model complexity. Specifically, the yolov8X cls algorithm and the above training set are used for model training, which makes the model more complex, deeper and larger in size, and has better classification accuracy.

[0053] S105: If it is detected that the recognition results of the two models of the original image are inconsistent, the original image is used to expand the target sample set.

[0054] Specifically, after the sample collection system inputs the original image into the first detection model and the second detection model respectively, each model will output the recognition result of each transport object in the original image, that is, the category of the transport object. The sample collection system detects whether the recognition results of the transport object at the same position in the two models are consistent. If not, the original image of the input model is used to expand the target sample set. For example, assuming in a specific embodiment, for the same original image, at position A of the image, the first detection model recognizes that the transport object at A is a red plastic bottle, and the second detection model recognizes that the transport object at A is a green plastic bottle, then it is determined that the recognition results of the two models of the original image are inconsistent (red plastic bottles and green plastic bottles need to be classified).

[0055] It should be noted that the expanded target sample set is used to tune and train the first detection model and the second detection model to improve the accuracy of the first detection model and the second detection model in object category recognition. The specific training process will not be repeated here.

[0056] In the above embodiment, the sample collection system obtains the image information of the transported object on the conveyor belt through the sorting device end, and then changes the shooting angle of the transported object on the conveyor belt through the reflux device, thereby realizing multi-angle shooting of the transported object. At the same time, for the original pictures taken, by comparing the recognition results of the first detection model and the second detection model, valuable sample original pictures are screened out to expand the target sample set, thereby improving the comprehensiveness and accuracy of target sample collection. In addition, through the cooperation of the sorting device end and the reflux device, the process of automatically collecting target samples is realized, which reduces manual participation and significantly improves the efficiency and quality of sample collection.

[0057] The following is a more detailed description of the process of the method provided by this embodiment. Figure 2 As shown, it is another flow chart of a method for improving sorting accuracy of a sorting line based on an AI algorithm in an embodiment of the present application.

[0058] S201, matching the height difference value corresponding to the transported object from the database, and adjusting the height difference between the conveyor belt and the reflow device.

[0059] During the sample collection process, the transported objects come in various shapes and sizes. In order to allow objects of different sizes to fall smoothly from the conveyor belt to the reflow device and achieve a change in angle when re-entering the conveyor belt, the height difference between the conveyor belt and the reflow device needs to be dynamically adjusted according to the characteristics of the object.

[0060] Specifically, the sample collection system stores the size information of various common transport objects and their corresponding optimal height difference values ​​in the database in advance. When the transport object for sampling is determined, the system will automatically search for the information of the object in the database and match the corresponding height difference value. Then, based on this value, the control module adjusts the distance between the conveyor belt and the reflux device in real time through actuators such as servo motors or hydraulic devices to ensure that the transport object will not be unable to achieve angle change due to too small a height difference during the process of falling and re-entering the conveyor belt, nor will it be damaged or rebound outside the conveyor belt due to too large a height difference.

[0061] For example, if the current sample to be collected is a small can, the database matches the optimal height difference to be 10cm. The sample collection system will control the actuator to raise the conveyor belt or lower the reflow device until the height difference between the two reaches 10cm. When switching to a larger carton sample, the matched height difference may be 2m, and the sample collection system will increase the distance between the conveyor belt and the reflow device accordingly. This adaptive adjustment mechanism ensures that different transported objects can obtain ideal angle changes during the reflow process, thereby capturing a variety of images.

[0062] In the above embodiment, the sample collection system introduces a mechanism for dynamically adjusting the height difference between the conveyor belt and the reflux device. By matching the height difference value corresponding to the transported object from the database to adjust the height, it is possible to handle objects of various sizes and shapes, so that the landing point of the transported object can be changed during the process of falling from the conveyor belt to the reflux device, so that when the reflux device re-enters the conveyor belt for shooting, original pictures at different angles can be obtained, thereby improving the quality of sample collection.

[0063] S202: Acquire the original image of the transport object through the sorting equipment end and the return device.

[0064] This step has been introduced in the above steps S102 to S103 and will not be repeated here.

[0065] S203: When it is detected that the number of reflux times exceeds a preset cycle threshold, the transport object dropped on the reflux device is no longer sent back to the conveyor belt for shooting.

[0066] Although the reflow device can improve the sampling efficiency of transported objects, if the same object is allowed to circulate between the conveyor belt and the reflow device indefinitely, it will not only occupy system resources, but also the sampling value of the image will decrease after the transported object is photographed multiple times. Therefore, the sample collection system sets a preset cycle threshold to control the number of reflows of the transported object.

[0067] Specifically, the sample collection system tracks and records the number of reflows of the same batch of transported objects (the first time they are placed on the conveyor within a preset time period). When the same batch of transported objects falls from the conveyor to the reflow device for the first time, its corresponding counter is initialized to 1. Every time the object reenters the conveyor and is photographed again, the counter is incremented by 1. Once the count reaches the preset cycle threshold (for example, 3 or 5 times), the control module sends an instruction to the actuator to direct the objects on the reflow device into the collection box instead of continuing to send them back to the conveyor through the reflow device. In this way, the system can strike a balance between obtaining sample images from a sufficient number of perspectives and avoiding unnecessary repeated sampling.

[0068] S204: Identify first category information and coordinate information of each transport object in the original image using a first detection model.

[0069] Specifically, after the sample collection system obtains the original image (the first original image or the second original image) of the transport object through the camera of the sorting device client, the original image is input into the first detection model to identify the first category information and coordinate information of each transport object in the original image. Figure 44 is a schematic diagram of an exemplary scene in which the first detection model mounted on the optical sorter in the embodiment of the present application recognizes an original image. 41 is a target image in the original image. The first detection model recognizes the transport object in the original image and obtains a plurality of square target images ( Figure 4 There are 6 target images in the image), and each target image is numbered and classified. The first category information and coordinate information of each transport object are obtained.

[0070] S205 , intercepting a target image corresponding to each transport object from the original image and converting it into a three-channel image.

[0071] After step S204, the sample collection system crops the original image and captures the target image in the original image. Then, the captured target image is scaled down to a three-channel image in BGR format with a width and height of 640*640 by using a padding method.

[0072] It should be noted that the padding method refers to adding a certain number of pixels around the edge of the image so that the size of the processed image meets specific requirements. It is a commonly used preprocessing method in the field of computer vision and image processing and is not limited here.

[0073] In addition, the captured target image size may be in other formats, such as 112*112BGR, 224*224BGR, and 320*320BGR, in addition to being scaled to a 640*640BGR format in width and height, which is not limited here.

[0074] S206: Identify second category information of the three-channel image using a second detection model.

[0075] Specifically, the sample collection system uploads the three-channel image obtained in step S205 to the cloud server, and the second detection model carried by the cloud server recognizes the received three-channel image to determine the category of the transport object in the three-channel image, that is, the second category information.

[0076] In the above embodiment, the sample collection system captures an image of a single transport object from the original image as the target image, converts it into a three-channel image in a preset format, and then inputs it into the second detection model for recognition. By preprocessing the target image, the influence of background noise can be reduced, so that the second detection model can focus more on the characteristics of the target object, thereby improving the recognition accuracy and facilitating the verification of the recognition result of the first model.

[0077] S207: Whether the first category information is the same as the second category information.

[0078] It is understandable that each original image can be identified as having one or more target images in the first detection model. When the first category information and the second category information of the same target image are different, the two model recognition results are determined to be inconsistent, and the process proceeds to step S209. Otherwise, the two model recognition results are determined to be consistent, and the process proceeds to step S208.

[0079] For example, in a specific embodiment, glass products and plastic products need to be classified. Assume that at a certain moment, an original image M is obtained by the sorting device. The first detection model identifies the transport object (target image) at the position (234, 267) of the original image M as a glass product. The target image is then uploaded to the second detection model for identification, and it is found that the transport object in the target image is a plastic product. Then, the sample collection system determines that the first category information is different from the second category information.

[0080] S208: Determine whether the recognition results of the two models are consistent.

[0081] If the recognition results of the two models are determined to be consistent, it means that the first detection model on the sorting device end is accurate, and the original image input to the model will not affect the recognition result of the model. Therefore, it is determined that the sampling value of the original image is low and is not used for the expansion of the target sample set.

[0082] S209: Call the SAM segmentation algorithm to automatically annotate the original image and generate annotated data.

[0083] If the recognition results of the two models are inconsistent, it means that the first detection model on the sorting device side or the second detection model on the cloud server is inaccurate, and the original image input to the model affects the recognition result of the model. Therefore, it is determined that the sampling value of the original image is higher and is used to expand the target sample set, thereby improving the recognition accuracy of the model (the first detection model and the second detection model).

[0084] Furthermore, in order to add the original image to the target sample set, the original image needs to be preprocessed first. Specifically, the sample collection system calls the SAM segmentation algorithm to automatically annotate the original image and generate annotation data. This reduces the manual workload and improves the efficiency of annotation.

[0085] It should be noted that the SAM algorithm uses the self-attention mechanism of Transformer to establish long-distance dependencies between different areas of the image, thereby achieving accurate segmentation of the target object. I will not go into details here.

[0086] S210: Expand the target sample set using the manually fine-tuned labeled data.

[0087] Specifically, after step S209, the sample collection system can display the original image and its corresponding annotation data on the display. At this time, the user can interact with the sample collection system on the display to modify the annotation data of the original image, thereby making up for the possible defects of the SAM segmentation algorithm. Then, the target sample set is expanded based on the manually fine-tuned annotation data to achieve the acquisition of the sample image.

[0088] In the above embodiment, the sample collection system automatically annotates the original image by introducing the SAM segmentation algorithm, and generates high-quality valid annotation data in combination with the adjustment of the technician. This human-machine combination not only improves the efficiency of annotation, but also ensures the accuracy of annotation. Automatic annotation reduces the manual workload, while manual adjustment makes up for the defects that may exist in the automatic algorithm. This method can not only quickly expand the high-quality sample set, but also continuously improve the recognition ability of the system, providing strong support for the iterative optimization of the model.

[0089] S211: Input the newly added valid labeled data into the first detection model and the second detection model for tuning and training.

[0090] Specifically, the annotated data after manual fine-tuning is valid annotated data. The sample collection system adds the valid annotated data to the target sample set, and at the same time counts the number of valid annotated data, that is, the data increment of the target sample set. When it is detected that the data increment exceeds the data increment threshold set by the user, the newly added valid annotated data is input into the first detection model and the second detection model for tuning training, and the model parameters are adjusted according to the training results, thereby improving the accuracy of the model in identifying objects. The specific tuning training process is basically the same as the training process of the model in step S104, and will not be repeated here.

[0091] In the above embodiment, the sample collection system can timely utilize the newly added valid annotated data to improve the model performance by setting a threshold and performing model tuning training after the threshold is reached. This mechanism ensures that the model can continuously learn new sample features and adapt to the ever-changing environment.

[0092] The sample collection system of the embodiment of the present invention is applied to electronic equipment. Figure 5 A schematic diagram of the architecture of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0093] It should be noted that Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0094] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions (computer programs), or by controlling related hardware through instructions (computer programs), and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. The electronic device of this embodiment includes a storage medium and a processor, wherein a plurality of instructions are stored in the storage medium, and the instructions can be loaded by the processor to execute any step of the method provided in the embodiment of the present invention.

[0095] Specifically, the storage medium and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more signal lines. The storage medium stores computer execution instructions for implementing the data access control method, including at least one software function module that can be stored in the storage medium in the form of software or firmware. The processor executes various functional applications and data processing by running the software program and module stored in the storage medium. The storage medium can be, but is not limited to, random access storage medium (Random Access Memory, referred to as: RAM), read-only storage medium (Read Only Memory, referred to as: ROM), programmable read-only storage medium (Programmable Read-Only Memory, referred to as: PROM), erasable read-only storage medium (Erasable Programmable Read-Only Memory, referred to as: EPROM), electrically erasable read-only storage medium (Electric Erasable Programmable Read-Only Memory, referred to as: EEPROM), etc. Among them, the storage medium is used to store programs, and the processor executes the program after receiving the execution instruction.

[0096] Furthermore, the software programs and modules in the above-mentioned storage medium may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components. The processor may be an integrated circuit chip having signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., which may implement or execute the various methods, steps, and logic flow diagrams disclosed in this embodiment. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0097] Since the instructions stored in the storage medium can execute the steps in any method provided in the embodiments of the present invention, the beneficial effects of any method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0098] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for improving sorting accuracy based on AI algorithm sorting line, applied to sample collection system, characterized in that: The method comprises: determining a transport object for target sample collection, wherein the transport object is placed on a conveyor belt; Acquiring a first original image of the transport object through a sorting device end, wherein a camera for photographing is installed on the sorting device end; The transport object dropped on the reflow device is put back onto the conveyor belt for shooting to obtain a second original picture, wherein there is a height difference between the reflow device and the end of the conveyor belt, which is used to change the contact position between the transport object and the conveyor belt; Inputting the original image into the first detection model and the second detection model respectively, and judging whether the recognition results of the two models are consistent, the original image includes the first original image and the second original image, the first detection model is installed on the sorting device end, and the second detection model is installed on the cloud server; If it is detected that the recognition results of the two models of the original image are inconsistent, the target sample set is expanded using the original image.

2. The method according to claim 1, characterized in that The step of inputting the original image into the first detection model and the second detection model respectively, and determining whether the recognition results of the two models are consistent specifically includes: Inputting the original image into the first detection model to identify first category information and coordinate information of each transport object in the original image; Inputting the original image into a second detection model to identify second category information of the transport object corresponding to each coordinate information; If it is detected that the first category information is the same as the second category information, then it is determined that the recognition results of the two models are consistent; If it is detected that the first category information is different from the second category information, it is determined that the two model recognition results are inconsistent.

3. The method according to claim 1, characterized in that The step of inputting the original image into the second detection model to identify the second category information of the transport object corresponding to each coordinate information specifically includes: Intercepting a target image corresponding to each transport object from the original image, wherein the position of the target image is determined according to the coordinate information; Converting the target image into a three-channel image in a preset format; The three-channel image is input into a second detection model to identify second category information corresponding to the three-channel image.

4. The method according to claim 1, characterized in that After the step of determining the transport object for target sample collection, the method further includes: Matching the height difference value corresponding to the transport object from a database; The height difference between the conveyor belt and the reflow device is adjusted according to the height difference value.

5. The method according to claim 1, characterized in that If it is detected that the recognition results of the two models of the original image are inconsistent, the step of using the original image to expand the target sample set specifically includes: Call the SAM segmentation algorithm to automatically annotate the original image and generate annotation data; Receiving adjustment parameters of the labeled data by a technician; Fine-tuning the annotation data according to the adjustment parameters to obtain valid annotation data of the original image; The target sample set is expanded using the valid labeled data.

6. The method according to claim 5, characterized in that After the step of using the original image and the effective annotation data to expand the target sample set, the method further includes: Receive the data increment threshold set by the user; After detecting that the amount of valid annotated data in the target sample set exceeds the data increment threshold, the newly added valid annotated data is input into the first detection model and the second detection model for tuning training.

7. The method according to claim 1, characterized in that After the step of sending the transported object dropped on the reflow device back onto the conveyor belt for photographing to obtain a second original picture, the method further includes: Count the number of return flows of transported objects; When it is detected that the number of reflows exceeds a preset cycle threshold, the transported object dropped on the reflow device is no longer sent back onto the conveyor belt for photographing.

8. A sample collection system, characterized in that: The sample collection system includes: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the sample collection system to perform the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a sample collection system, the sample collection system is caused to perform the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is executed on a sample collection system, the sample collection system is caused to perform the method according to any one of claims 1 to 7.

Images