Automated screening method and apparatus based on image analysis

Abstract

The application provides an automatic screening method and device based on image analysis, which uses a bearing plate as a bearing carrier of screening objects, selects corresponding processing devices according to two bearing states of bearing grooves in the bearing plate to realize screening of target objects: when the bearing plate is partially loaded, a first processing device is used, a movable first lighting device is used to irradiate the bearing grooves with screening objects one by one, and a receiving plate is used to collect local images; when the bearing plate is fully loaded, a second processing device is used, global illumination is used to directly collect global images of the bearing plate, and different solutions for different states are realized; and then a preset quantity and light transmittance comparison table is used to analyze the collected images to identify target objects in each bearing groove and the quantity of the target objects in each bearing groove, so that the application realizes automatic screening of target objects and solves the problems of strong dependence on manual operation and low efficiency in traditional screening.

Description

TECHNICAL FIELD

[0001] The present application relates to data processing technology, and in particular to an automatic screening method and device based on image analysis. BACKGROUND

[0002] Cocoon is the core product in silkworm production, and its quality directly affects the efficiency of subsequent reeling process and silk quality. The hatched cocoon (i.e. the empty cocoon left after the silkworm pupa hatches) cannot meet the needs of reeling production because of its loose internal structure and damaged silk integrity, and needs to be accurately screened and removed from batches of cocoon before processing.

[0003] In existing cocoon screening operations, manual screening methods are mostly used, which rely on operators to observe the appearance and feel of the cocoon by naked eye or to make judgments with the aid of simple light assistance. This method requires a large amount of labor cost, and the screening efficiency is greatly affected by the experience and physical condition of the personnel, making it difficult to meet the needs of large-scale production.

[0004] Therefore, how to realize the automatic screening of hatched cocoon has become a key problem to be solved. SUMMARY

[0005] The present application provides an automatic screening method and device based on image analysis, which can realize the automatic screening of hatched cocoon.

[0006] In a first aspect of the present application, an automatic screening method based on image analysis is provided, comprising:

[0007] A processing device for determining the screening object based on the bearing image of the bearing plate, the processing device comprising a first processing device and a second processing device;

[0008] When the processing device is the first processing device, a first lighting device is called to irradiate and image collect the screening object in each bearing slot on the bearing plate in turn, and a local image is obtained;

[0009] When the processing device is the second processing device, a second lighting device is called to irradiate and image collect the entire bearing plate, and a global image is obtained;

[0010] Image analysis is performed on the local image or global image to obtain a target object.

[0011] Optionally, in a possible implementation manner of the first aspect,

[0012] The processing device for determining the screening object based on the bearing image of the bearing plate, the processing device comprising a first processing device and a second processing device, comprises:

[0013] The bearing image of the bearing plate is acquired based on the front image acquisition unit, and the bearing state of each bearing slot on the bearing plate is identified according to the bearing image.

[0014] When it is determined that the screening objects are in the bearing slots, mark the bearing state of the corresponding bearing slot as a full load state, and take the bearing slot in the full load state as a full load slot, and count the number of full load slots as the bearing number;

[0015] Obtain the number of bearing slots on the bearing plate as the full load number, compare the full load number with the bearing number, and when the full load number is greater than the bearing number, select the first processing device as the processing device of the bearing plate;

[0016] When the full load number is equal to the bearing number, the second processing device is selected as the processing device of the bearing plate.

[0017] Optionally, in a possible implementation manner of the first aspect,

[0018] When the determination processing device is the first processing device, the first illumination device is called to sequentially irradiate the screening objects in each bearing slot on the bearing plate and perform image acquisition to obtain a local image, including:

[0019] Move the bearing plate into the first processing device, obtain the bearing profile of each bearing slot based on the bearing image, and determine the profile center of the bearing profile as the bearing center;

[0020] Coordinate the bearing image to determine the coordinates of the bearing center as the bearing position of the bearing slot;

[0021] Obtain the bearing position of each full load slot as a full load position, and call the first illumination device above the bearing plate to sequentially irradiate each full load slot based on the full load position, and acquire the image of the receiving plate below the bearing plate as a local image.

[0022] Optionally, in a possible implementation manner of the first aspect,

[0023] The first illumination device above the bearing plate is called to sequentially irradiate each full load slot based on the full load position, and the image of the receiving plate below the bearing plate is acquired as a local image, including:

[0024] Based on the first image acquisition device in the receiving area, a light spot mapped from the transmission hole at the bottom of the full load slot to the receiving plate is acquired, the receiving area is the area between the receiving plate and the bearing plate, and the light spot is located in the receiving slot on the receiving plate which is symmetrical to the bearing slot;

[0025] When it is determined that each first illumination device is located at the full load position, the first image acquisition device is enabled to acquire the image of the light spot on the receiving plate to obtain the local image corresponding to the screening object in the full load position.

[0026] Optionally, in a possible implementation manner of the first aspect,

[0027] When the judgment processing device is the second processing device, the second lighting device is triggered to irradiate the entire carrier plate and image collection is performed to obtain a global image, including:

[0028] The carrier plate is moved into the second processing device, and the second lighting device located below the carrier plate irradiates the screening objects in the entire carrier plate through the transmission holes in the bottom of each carrier groove;

[0029] The second image collection device above the carrier plate obtains a global image of the screening objects on the carrier plate.

[0030] Optionally, in a possible implementation manner of the first aspect,

[0031] The analysis of the local image or the global image to obtain the target object includes:

[0032] The number of screening objects in each carrier groove is determined based on the carrier image to obtain an actual loading number;

[0033] A preset pixel point number range corresponding to the actual loading number is obtained from a preset number-transmittance table according to the actual loading number;

[0034] The target object is obtained by comparing the local image or the global image with the preset pixel point number range.

[0035] Optionally, in a possible implementation manner of the first aspect,

[0036] The comparison of the local image or the global image with the preset pixel point number range to obtain the target object includes:

[0037] The receiving contour of the receiving groove is obtained according to the local image, and a contour center of the receiving contour is determined as a receiving center;

[0038] A coordinate system is established with the same coordinate origin as the carrier image for the local image, and a coordinate of the receiving center is obtained as a receiving position of the receiving groove;

[0039] A corresponding receiving position in the local image is determined as a detection position based on the actual loading position corresponding to the carrier image;

[0040] The local image is subjected to binaryzation processing, and a number of white pixel points at each detection position is obtained as a detection number;

[0041] The target number of the target object at the corresponding actual loading position is obtained by comparing the detection number with the preset pixel point number range, and an actual loading position with a target number greater than or equal to 1 is determined as a target position;

[0042] The target object is determined based on the target position.

[0043] Optionally, in a possible implementation manner of the first aspect,

[0044] The comparison of the local image or the global image with the preset pixel point number range obtains the target object, and the comparison includes:

[0045] The global image is used to obtain the bearing contour of each bearing groove, the contour center of the bearing contour is determined as an analysis center, the global image is subjected to coordinate processing, and the coordinates of the analysis center are determined as an analysis position of the bearing groove;

[0046] The global image is subjected to binary processing, and the number of white pixel points at each analysis position is obtained as an analysis number;

[0047] The analysis number is compared with the preset pixel point number range, a target number of the target object at the corresponding analysis position is obtained, and an analysis position with a target number greater than or equal to 1 is determined as a target position;

[0048] The target object is determined based on the target position.

[0049] Optionally, in a possible implementation manner of the first aspect,

[0050] The determination of the target object based on the target position further includes:

[0051] In response to the grabbing information, a target position with an actual loading number equal to a target number is selected as a grabbing position, and a target position with an actual loading number greater than a target number is selected as an adjustment position;

[0052] Coordinates corresponding to the grabbing position are sent to a grabbing device, and the target object is obtained;

[0053] Coordinates corresponding to the adjustment position are obtained and sent to an adjustment end, and the target object with the target number is selected based on the adjustment end.

[0054] In a second aspect, the application provides an automatic screening device based on image analysis, which includes:

[0055] A distribution module is configured to determine a processing device of a screening object based on a bearing image of a bearing plate, and the processing device includes a first processing device and a second processing device;

[0056] A first irradiation module is configured to, when the processing device is the first processing device, call a first lighting device to irradiate and collect images of the screening object in each bearing groove of the bearing plate in sequence, and obtain a local image;

[0057] A second irradiation module is configured to, when the processing device is the second processing device, call a second lighting device to irradiate and collect images of the entire bearing plate, and obtain a global image;

[0058] an analysis module configured to perform image analysis on the local image or the global image to obtain the target object.

[0059] The present application has the following advantages:

[0060] 1. The present application acquires a bearing image of the bearing plate based on the pre-image acquisition unit, identifies the bearing state of each bearing groove and counts the bearing quantity. After comparing the full load quantity, the partially loaded bearing plate is moved into the first processing device, and the fully loaded bearing plate is moved into the second processing device. The first processing device irradiates the bearing grooves loaded with the screening objects (unscreened cocoon) one by one through the movable first lighting device, and obtains the local image in combination with the receiving plate and the first image acquisition device. The second processing device irradiates the entire bearing plate through the second lighting device, and obtains the global image through the second image acquisition device. The two processing devices are adapted to different bearing scenes, avoiding invalid detection of unloaded bearing grooves when partially loaded, and also avoiding inefficient one-by-one detection when fully loaded, thereby realizing the adaptation of screening of the target object (emerged cocoon) under different bearing states.

[0061] 2. The present application guarantees screening under different scenes through the two processing devices. The first processing device adopts the structure of upward movable lighting and downward receiving plate: the first lighting device can move to irradiate the loaded groove, and the light passes through the screening object and the transmission hole to form a light spot in the corresponding receiving groove of the downward receiving plate, and the first image acquisition device collects the light spot image (local image) reflecting the light transmittance of the screening object. The second processing device adopts the structure of downward fixed global lighting and upward image acquisition: the second lighting device is fixed below the bearing plate, and the lighting range covers the entire bearing plate. The light passes through the transmission hole at the bottom of each bearing groove to act on the screening object, and the second image acquisition device above the bearing plate can obtain the transmission image (global image) of the screening object in all bearing grooves at one time. The two structures are respectively suitable for partially loaded and fully loaded scenes, and ensure that high-quality transmission images can be collected by different processing devices, thereby providing data support for subsequent emergence determination.

[0062] 3. The present application determines the loaded quantity of each bearing groove based on the bearing image, and retrieves the preset pixel point quantity range corresponding to the loaded quantity from a preset transmittance comparison table. The comparison table is established through a large number of sample experiments, and covers the transmittance of different loaded quantities and different emergence combinations. When analyzing the local or global image, the corresponding preset pixel point quantity range is taken as the reference, the white pixel point quantity of each position is obtained through binary processing, and the target quantity and target position are obtained, thereby avoiding misjudgment caused by quantity difference of a single transmittance standard. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The application scenario of the technical solution provided by the present application is shown in the figure;

[0064] Figure 2 A flowchart illustrating an automated screening method based on image analysis provided by this invention;

[0065] Figure 3 This is a schematic diagram of the structure of the bearing plate in this invention;

[0066] Figure 4 This is a schematic diagram of the structure of the first processing device in this invention;

[0067] Figure 5 This is a schematic diagram of the structure of the second processing device in this invention;

[0068] Figure 6 This is a schematic diagram of the structure of an automated screening device based on image analysis provided by the present invention. Detailed Implementation

[0069] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0070] like Figure 1 The diagram illustrates a scenario of the technical solution provided by this invention. This application scenario includes: a server, a front-end acquisition unit, a first processing device, a second processing device, a gripping device, and an adjustment terminal. The server is communicatively connected to the front-end acquisition unit, the first processing device, the second processing device, the gripping device, and the adjustment terminal. The server controls the front-end image acquisition unit to acquire images of the carrier plate and determines the assigned processing device (either the first or second processing device) based on the acquired image of the carrier plate (see [reference]). Figure 2 After the screening of the screening items is completed in the first processing device, the server sends the position information of each carrying slot to the gripping device and the adjustment end. The gripping device directly grips the target item, and the adjustment end sorts the target item and the screening items.

[0071] This invention provides an automated screening method based on image analysis, comprising:

[0072] S1, a processing device for determining screening materials based on the carrying image of the carrier plate, the processing device including a first processing device and a second processing device.

[0073] It should be noted that cocoon is the core product in silkworm production, and its quality directly affects the efficiency of subsequent reeling process and silk quality. The pupated cocoon (i.e. the empty cocoon left after the silkworm pupa hatches) cannot meet the demand of reeling production due to its loose internal structure and damaged silk integrity, and needs to be accurately screened and removed from the batch of cocoon before processing. In the existing cocoon screening operation, manual screening method is mostly used, which relies on the operator to observe the appearance and touch of the cocoon by naked eye or to make a simple light assisted judgment. This method needs to invest a lot of labor cost, and the screening efficiency is greatly affected by the experience and physical condition of the personnel, which is difficult to meet the demand of large-scale production.

[0074] Therefore, the present application uses a bearing plate as the bearing carrier of the cocoon (i.e. the screening object), and selects the corresponding processing device according to the two bearing states of the bearing groove in the bearing plate in combination with the actual operation situation to realize the screening of the pupated cocoon (i.e. the target object). The first processing device is used when the bearing plate is partially loaded, the bearing groove with the screening object is irradiated one by one by the movable first lighting device, and the local image is collected by the receiving plate; the second processing device is used when the bearing plate is fully loaded, the global image of the bearing plate is collected directly by the global lighting, realizing different solutions for different actual situations. In combination with the preset quantity transmittance comparison table, the collected image is analyzed to identify whether the screening object (cocoon) in each bearing groove is the target object (pupated cocoon) and how many target objects are in each bearing groove. The present application realizes efficient and accurate screening of pupated cocoon, solves the problem of strong dependence on manual screening and low efficiency in traditional screening, and meets the demand of large-scale cocoon processing.

[0075] It can be understood that before automatic screening, the screening object to be screened needs to be placed on the bearing plate. As shown in Figure 3 The bearing plate has bearing grooves for bearing the screening object, so the screening object falls into the bearing groove. If there are screening objects scattered on the surface of the bearing plate, other sweeping tools can be used to push the screening objects on the surface into the bearing groove or out of the bearing plate to wait for the next batch of screening. First, the specific bearing state of each bearing groove on the bearing plate (i.e. whether the screening object is placed in each bearing groove) is obtained by image recognition and other means. Then, the bearing states of all bearing grooves are statistically analyzed to determine whether the current bearing plate belongs to "partial loading" (there are some bearing grooves without screening objects) or "full loading" (all bearing grooves have screening objects). Finally, different processing devices are determined according to the two situations of "partial loading" and "full loading", so that the subsequent screening process can be adapted to the actual state of the bearing plate, and the screening accuracy and efficiency are improved.

[0076] The screening object refers to cocoon that has not been screened, including target objects and non-target objects, the target object refers to cocoon that has been hatched, and the non-target object refers to cocoon that has not been hatched. The bearing plate refers to a flat plate structure for batch bearing and conveying the screening object, the size of which is designed according to the production line requirements, and the surface is flat to ensure smooth transmission. The bearing groove refers to a groove structure opened on the bearing plate, which is specifically used for placing the screening object, and the shape of each bearing groove is usually a cylinder, the diameter of which is slightly larger than the average length of the screening object (the average length of cocoon is about 3-5 cm), and the depth can be designed to be 5-8 cm, which can ensure stable placement of the screening object and avoid stacking too high to affect detection. The present application is based on light screening of the screening object, so the bottom of each bearing groove is provided with a transmission hole for light transmission. The bearing state is used to describe whether the screening object is placed in the bearing groove, and is specifically divided into: full load state (one or more screening objects are placed in the bearing groove) and empty load state (no screening object is placed in the bearing groove). The processing device refers to a special device for detecting and screening the screening object, and two types of devices are designed in the present application to adapt to different bearing states: the first processing device is suitable for the bearing plate with “partial loading”, can position the bearing groove loaded with the screening object for directional detection, has high detection accuracy, and is suitable for processing the case where there is an empty bearing groove; the second processing device is suitable for the bearing plate with “full loading”, can realize global illumination of the bearing plate, and can detect all bearing grooves, has high detection efficiency, and is suitable for processing the case where there is no empty bearing groove.

[0077] In some embodiments, the step S1 (determining the processing device of the screening object based on the bearing image of the bearing plate, the processing device including a first processing device and a second processing device) includes S11-S14:

[0078] S11, acquiring the bearing image of the bearing plate based on the front image acquisition unit, and identifying the bearing state of each bearing groove on the bearing plate according to the bearing image.

[0079] It can be understood that, in order to conveniently and accurately acquire the bearing state of each bearing groove, the front image acquisition unit is designed to complete the state judgment before the bearing plate enters the processing device. Specifically, a clear bearing plate image (i.e. the bearing image) is acquired by the front image acquisition unit, and the server can conveniently identify whether each bearing groove in the bearing image has a screening object, such as the image processing algorithm in the prior art.

[0080] The front image acquisition unit refers to an image acquisition device installed in front of the entrance of the processing device, which is usually an industrial camera. The bearing image is an image of the bearing plate taken by the front image acquisition unit, which is usually a complete image of the bearing plate taken by the front image acquisition unit at a top-down angle, and the bearing image presents the outline and internal state of the bearing groove.

[0081] S12, when it is determined that the screening object is in the bearing groove, marking the bearing state of the corresponding bearing groove as a full load state, and taking the bearing groove in the full load state as a full load groove, and counting the number of full load grooves as the bearing number.

[0082] It can be understood that based on the image analysis result of step S11, the bearing groove in which the screening object is confirmed to exist is marked with a "full load state" mark to facilitate positioning in subsequent steps, and the total number of bearing grooves marked as a full load state is defined as the bearing number.

[0083] Among them, the full load groove refers to the bearing groove marked as a full load state, and the bearing number refers to the number of full load grooves.

[0084] S13, taking the number of bearing grooves on the bearing plate as the full load number, comparing the full load number with the bearing number, and determining that when the full load number is greater than the bearing number, the first processing device is selected as the processing device of the bearing plate.

[0085] It is not difficult to understand that the number of bearing grooves on the current bearing plate is retrieved from the system preset parameters, or the number of bearing grooves on the current bearing plate is identified through the bearing image, and this number is taken as the full load number. When the full load number is greater than the bearing number (that is, there are bearing grooves without screening objects), the first processing device is selected as the processing device of the bearing plate. In the first processing device, the first lighting device will irradiate each full load groove one by one.

[0086] Among them, the full load number refers to the total number of bearing grooves on the bearing plate, which can be a fixed parameter of the bearing plate (such as 24 grooves, 36 grooves, 48 grooves, etc.), determined by production design and pre-stored in the system database.

[0087] S14, when it is determined that the full load number is equal to the bearing number, the second processing device is selected as the processing device of the bearing plate.

[0088] It is not difficult to understand that when it is determined that the full load number is equal to the bearing number (that is, all bearing grooves have screening objects), the second processing device is selected as the processing device of the bearing plate, and in the second processing device, the second lighting device will irradiate the entire bearing plate.

[0089] S2, when it is determined that the processing device is the first processing device, the first lighting device is retrieved to irradiate and collect images of the screening objects in each bearing groove on the bearing plate one by one, and the local image is obtained.

[0090] It can be understood that when the bearing plate enters the first processing device, the position information of each full load groove needs to be combined to irradiate each full load groove one by one using the first lighting device, and obtain the local image corresponding to each full load groove.

[0091] It should be noted that, for example,Figure 4 As shown, in the first processing device of the present application, the first lighting device is a movable lighting device, which can be driven by a motor to move on the slide rail. The first lighting device is arranged on the top of the first processing device, because not all of the bearing slots on the bearing plate in the first processing device are equipped with the screening object, so the first lighting device moves on the top of the first processing device and positions the lighting. After the light penetrates through the screening object and the transmission hole at the bottom of the bearing slot, corresponding light spots (light points) are formed on the receiving plate below the bearing plate. The size of the receiving plate is the same as that of the bearing plate, and the light spots correspond to the positions of the bearing slots. Then, the image acquisition device synchronously acquires the light spot images.

[0092] In some embodiments, the step S2 (when the processing device is determined to be the first processing device, the first lighting device is called to sequentially irradiate the screening object in each bearing slot of the bearing plate and image acquisition is performed to obtain the local image) comprises S21-S23:

[0093] S21, the bearing plate is moved into the first processing device, the bearing profile of each bearing slot is obtained based on the bearing image, and the profile center of the bearing profile is determined as the bearing center.

[0094] It can be understood that when the processing device is determined to be the first processing device, the bearing plate is moved into the first processing device by the transmission mechanism of the production line, and the first lighting device is determined in position before lighting. Therefore, the position of the bearing slot needs to be confirmed first. Specifically, when the bearing plate is moved into the first processing device, the bearing profile (usually circular, matching the design shape of the bearing slot) of each bearing slot is obtained based on the bearing image by using an edge detection algorithm (such as Canny operator, threshold adaptive adjustment). Then, the geometric center of the bearing profile is calculated and defined as the bearing center.

[0095] Herein, the bearing profile refers to the line profile of the bearing slot in the bearing image, specifically the closed line of the slot edge of the bearing slot, and the bearing center refers to the geometric center of the bearing profile.

[0096] S22, the bearing image is subjected to coordinate processing, and the coordinates of the bearing center are determined as the bearing position of the bearing slot.

[0097] It is not difficult to understand that the bearing image is subjected to coordinate processing to determine the physical position of each bearing slot. Specifically, the lower left corner or the center of the bearing plate in the bearing image can be selected as the coordinate origin to establish a two-dimensional coordinate system, the coordinate position corresponding to the bearing center is determined as the physical position of the bearing slot, and defined as the bearing position.

[0098] Herein, the bearing position refers to the coordinate position of the bearing slot, which corresponds to the center of the bearing slot.

[0099] S23, obtaining the carrying position of each real-load slot as a real-load position, calling the first lighting device above the carrying plate to irradiate each real-load slot in turn based on the real-load position, and collecting the image of the receiving plate below the carrying plate as a local image.

[0100] It can be understood that the first processing device is used to screen the screening objects in each real-load slot, and the carrying position corresponding to the real-load slot is screened out based on the carrying position obtained in step S22, and the carrying position is used as the real-load position. According to the distribution of the real-load position, the moving path of the first lighting device can be planned by using the "greedy algorithm" in the prior art, and each real-load slot is irradiated in turn, thereby improving the work efficiency.

[0101] Further, the first lighting device moves to each real-load position in turn according to the moving path, and the LED light source in the first lighting device is started after the position sensor confirms that the first lighting device reaches the real-load position (the light intensity is preset according to the average thickness of the screening object). After the light penetrates the screening object and the light transmission hole of the carrying slot, a light spot is formed in the corresponding groove (receiving slot) of the receiving plate below, and then the first image acquisition device above the receiving plate is triggered to shoot the light spot image. The image is the local image, and the brightness characteristics of each light spot directly reflect the light transmittance of the corresponding screening object.

[0102] It should be noted that in the embodiment, the number of the first lighting device can be set to one, or multiple first lighting devices can participate together. If multiple first lighting devices are used, the multiple first lighting devices are determined to reach the real-load position at the same time, and then irradiation and local image acquisition are performed.

[0103] In some embodiments, step S23 (based on the real-load position, calling the first lighting device above the carrying plate to irradiate each real-load slot in turn, and collecting the image of the receiving plate below the carrying plate as a local image) includes S231-S232:

[0104] S231, based on the first image acquisition device in the receiving area, collecting the light spot mapped from the transmission hole at the bottom of the real-load slot to the receiving plate, the receiving area being the area between the receiving plate and the carrying plate, and the light spot being located in the receiving slot on the receiving plate which is symmetrical to the carrying slot.

[0105] It should be noted that the receiving plate is located below the bearing plate for receiving the light spot transmitted from the bearing plate. Specifically, the receiving plate is provided with receiving grooves corresponding to the bearing grooves above, so as to receive the light emitted from the transmission hole and realize accurate convergence and positioning of the light spot. In this embodiment, firstly, the light must pass through the screening material in the bearing groove and the transmission hole at the bottom of the bearing groove in sequence to form a vertical downward directional propagation path, avoiding overflow from the side wall of the bearing groove; secondly, the space between the bearing plate and the receiving plate (i.e. the receiving area) is designed to be airtight and light shielding, completely blocking external light (such as workshop lighting and equipment indicator light), so as to ensure that only the effective light transmitted by the transmission hole exists in the receiving area; in addition, the receiving grooves on the receiving plate correspond to the bearing grooves on the bearing plate, and the center points of the two are on the same vertical line, so that the transmitted light converges in the receiving groove to form a regular light spot, avoiding the light spot from spreading to the outside area of the receiving groove. Further, the shape and size of the receiving plate can be the same as those of the bearing plate.

[0106] It can be understood that the first image acquisition device is installed in the receiving area, and the light spot on the receiving plate is acquired by the first image acquisition device after the first lighting device starts lighting.

[0107] The first image acquisition device can be an industrial camera, and the receiving groove refers to the groove on the receiving plate corresponding to the bearing groove. The physical structure (such as the inclined side wall of the groove body) of the receiving groove will constrain the divergent light in a single receiving groove and prevent the light from spreading to the adjacent area.

[0108] S232, when it is determined that each first lighting device is located at the bearing position, the first image acquisition device is enabled to acquire the image of the light spot on the receiving plate, and a local image corresponding to the screening material in the bearing position is obtained.

[0109] It should be noted that lighting is started only after it is ensured that all first lighting devices reach the bearing position. If lighting is started before a part of the first lighting devices reaches the bearing position, the light may directly irradiate the bearing groove without screening material, and the light directly passes through the transmission hole to form a light spot on the receiving plate, which may misjudge that there is a target object in the bearing groove.

[0110] It can be understood that lighting is started and the first image acquisition device is enabled to acquire the image of the light spot on the receiving plate only when it is determined that each first lighting device is located at the bearing position, and a local image corresponding to the screening material in the current bearing position is obtained.

[0111] After the acquisition is completed, the first lighting device can be moved to the next bearing position according to the movement path again for the next acquisition.

[0112] The local image refers to the image of the receiving plate corresponding to the current bearing position after irradiation, which records the state of the light spot corresponding to the current bearing position.

[0113] S3, when the judgment processing device is the second processing device, the second lighting device is called to irradiate the entire bearing plate and image collection is performed to obtain a global image.

[0114] It should be noted that, as shown in Figure 5 The second processing device is applicable to a "fully loaded" bearing plate and can irradiate the entire bearing plate and detect all bearing slots. In the second processing device, the second lighting device is located below the bearing plate and irradiates the entire bearing plate from below, keeping the area below the bearing plate a constant light. The transmission hole at this time plays a "classification screening" role: the bearing plate entity area blocks the light, and only the transmission hole allows the light to pass through, forming a concentrated light beam corresponding to each bearing slot, which accurately acts on the screening object in each bearing slot (no empty slot, no need to avoid), realizing accurate action of a single bearing slot under global irradiation. Because the second processing device irradiates the entire bearing plate, the light can only penetrate out of the transmission hole after irradiating the bearing plate, and the light beam with a certain range of diffusion can realize concentrated irradiation of the screening object in each bearing slot through the transmission hole, so this is one of the advantages of irradiating from below when irradiating uniformly. The global image is collected by the second image collection device above the bearing plate. The second processing device is for the "fully loaded" scenario, and all bearing slots have screening objects. The light penetrates the bearing slot and directly acts on the screening object, without the need for a receiving plate to concentrate light. This embodiment cancels the receiving plate on the one hand and adjusts the lighting method to fixed range lighting from mobile lighting.

[0115] The second lighting device refers to a device that irradiates the bearing plate in a fixed position in the second processing device.

[0116] In some embodiments, step S3 (when the judgment processing device is the second processing device, the second lighting device is called to irradiate the entire bearing plate and image collection is performed to obtain a global image) includes S31-S32:

[0117] S31, the bearing plate is moved into the second processing device, and the second lighting device located below the bearing plate irradiates the screening object in the entire bearing plate through the transmission hole at the bottom of each bearing slot.

[0118] It can be understood that when the determination processing device is the second processing device, the carrier plate is moved into the second processing device through the transmission mechanism of the production line, the second lighting device below the carrier plate is turned on to realize lighting, the position of the second lighting device is fixed, the second lighting device is arranged in the area below the carrier plate (not directly in contact with the carrier plate, which can be the bottom of the second processing device), and the illumination range of the second lighting device completely covers the carrier plate, and the illumination intensity of the entire carrier plate is consistent. The light irradiated by the second lighting device passes through the transmission holes in the bottom of each carrier groove to irradiate the screening objects in the entire carrier plate. In the bottom lighting scene, the transmission holes are not only light channels (only allowing light source light to pass through), but also can constrain the light to avoid the light from spreading to adjacent carrier grooves.

[0119] S32, acquiring a global image of the screening objects on the carrier plate based on a second image acquisition device above the carrier plate.

[0120] It can be understood that the second image acquisition device is arranged above the carrier plate, which can be the top of the second processing device, and the second image acquisition device can adopt an industrial camera. The global image refers to the image of the carrier plate acquired by the second image acquisition device after the second lighting device is turned on in the second processing device.

[0121] S4, analyzing the local image or the global image to obtain the target object.

[0122] It should be noted that in the screening process of the screening objects, if the screening is directly performed by using a single light transmittance threshold, the influence of the number difference of the screening objects in the carrier groove on the light transmittance will be ignored. For example, there is a difference between the light transmittance of 1 cocoon (short light penetration path) in the same carrier groove and the light transmittance of 2 stacked cocoons (long light penetration path). If the same light transmittance threshold is used, it may cause that when multiple cocoons are stacked, all the cocoons in the carrier groove are directly determined as non-emerged cocoons because the light transmittance does not meet the standard.

[0123] Therefore, it is necessary to determine the number of the screening objects in each carrier groove, and determine the number of the target objects in the carrier groove according to the light transmittance corresponding to the number of the screening objects.

[0124] In some embodiments, the step S4 (analyzing the local image or the global image to obtain the target object) includes S41-S43.

[0125] S41, determining the number of the screening objects in each carrier groove based on the carrier image to obtain the actual loading number.

[0126] It can be understood that the number of the screening objects in each carrier groove can be recognized according to the carrier image, and the actual number of the screening objects in a single carrier groove is taken as the actual loading number.

[0127] S42, obtaining the preset pixel point quantity range corresponding to the actual loading quantity from the preset quantity-transmittance contrast table according to the actual loading quantity.

[0128] It can be understood that the actual loading quantity of each bearing groove has been determined in step S41, and the preset quantity-transmittance contrast table in the system is called, and the preset pixel point quantity range in the quantity-transmittance contrast table is obtained according to the actual loading quantity of the single bearing groove. The preset pixel point quantity range can determine how many target objects in the screening object corresponding to the actual loading quantity.

[0129] The quantity-transmittance contrast table is a mapping relationship table of the actual loading quantity and the preset pixel point quantity range based on sample experiments. The preset pixel point quantity range represents a preset white pixel point quantity range for quantifying the quantity of target objects and non-target objects. The non-target object represents an un-hatched cocoon.

[0130] For example, the quantity-transmittance contrast table can be generated by the following steps:

[0131] Sample preparation: Select un-hatched cocoons and hatched cocoons of the same variety, and fill each bearing groove with “1 un-hatched cocoon”, “1 hatched cocoon”, “1 un-hatched cocoon, 1 hatched cocoon”, “2 un-hatched cocoons”, and “2 hatched cocoons”, respectively. Each group of samples has a quantity of ≥100;

[0132] Transmittance collection: Use the first / second processing device consistent with the production line to shoot local / global images, and count the white pixel point quantity range in each group of samples (such as 380-420 for 1 un-hatched cocoon).

[0133] Range determination: Take each group of ranges and add a safety redundancy as the preset pixel point quantity range (such as 350-450 for 1 un-hatched cocoon). The safety redundancy can be optimized and adjusted according to the verification results.

[0134] S43, comparing the local image or global image with the preset pixel point quantity range to obtain the target object.

[0135] It can be understood that after obtaining the local image or global image, the white pixel points at each bearing groove position in the image are compared with the preset pixel point quantity range to obtain the target object, which represents a hatched, hollow hatched cocoon.

[0136] The present application analyzes the local image and the global image respectively to obtain the target object.

[0137] In some embodiments, (comparing the local image or global image with the preset pixel point quantity range to obtain the target object) in step S43 includes A1-A6:

[0138] A1, determining a receiving profile of the receiving slot based on the local image, and determining a profile center of the receiving profile as a receiving center.

[0139] It can be understood that the edge profile of the receiving slot is extracted as the receiving profile based on the local image, and the geometric center of the receiving profile is taken as the receiving center.

[0140] A2, establishing a coordinate system with the same coordinate origin as the carrier image for the local image, and obtaining the coordinates of the receiving center as the receiving position of the receiving slot.

[0141] It can be understood that the receiving plate and the carrier plate are plate-shaped structures with the same shape and size, and the coordinate system is established with the same coordinate origin as the carrier image for the local image according to the symmetrical mapping relationship between the receiving plate and the carrier plate. For example, the coordinate system established in step S22 of the carrier image is established with the center of the carrier plate, and the coordinate system established in the embodiment of the local image is established with the center of the receiving plate. Based on the coordinate system established in the local image, the coordinates of the receiving center are taken as the receiving position of the receiving slot.

[0142] A3, determining the corresponding receiving position in the local image as the detection position based on the real load position corresponding to the carrier image.

[0143] It should be noted that the first illumination device is illuminated once, and a local image is obtained. Since the number of first illumination devices can be 1 or more, the number of light spots in each local image can be 1 or more, and the detection results of each light spot correspond to the real load position.

[0144] It can be understood that according to the coordinate system of the local image and the carrier image established in step A2, the real load position can be associated with the corresponding receiving position, that is, the real load position corresponds to the position of the light spot in the receiving plate, and the receiving position corresponding to the real load position is taken as the detection position.

[0145] A4, performing binaryzation processing on the local image, and obtaining the number of white pixel points at each detection position as the detection number.

[0146] It can be understood that the analysis of the transmittance of the screening material in the embodiment is the binaryzation processing of the local image in the prior art, which divides the image into white pixels (transmission area) and black pixels (non-transmission area), realizes the separation of the target and the background, and obtains the number of white pixel points at each detection position as the detection number.

[0147] A5, comparing the detection quantity with the preset pixel point quantity range, obtaining the target quantity of the target object at the corresponding real loading position, and taking the real loading position with the target quantity greater than or equal to 1 as the target position.

[0148] It can be understood that the white pixel point quantity (detection quantity) at the detection position is obtained through step A4. According to the corresponding real loading quantity at each detection position, the corresponding preset pixel point quantity range in the quantity transmittance comparison table is obtained, the detection quantity is compared with the preset pixel point quantity range, the target quantity of the target object at the corresponding real loading position is obtained, and the target quantity represents the quantity of the target object in a single bearing groove. For example, when the real loading quantity is 2, the preset pixel point quantity range corresponding to 1 cocoon and 1 un-hatched cocoon is 320-390, the preset pixel point quantity range corresponding to 2 hatched cocoons is 400-470, and the preset pixel point quantity range corresponding to 2 un-hatched cocoons is 260-300. When the detection quantity is 350, the hatching quantity at the corresponding real loading position is 1, and the real loading position is the hatching position.

[0149] A6, determining the target object based on the target position.

[0150] It should be noted that after the target position is obtained, the target object needs to be sorted out in combination with the target quantity.

[0151] In some other embodiments, the step S43 (comparing the local image or the global image with the preset pixel point quantity range to obtain the target object) includes B1-B4.

[0152] B1, obtaining the bearing contour of each bearing groove based on the global image, determining the contour center of the bearing contour as the analysis center, performing coordinate processing on the global image, and determining the coordinates of the analysis center as the analysis position of the bearing groove.

[0153] It can be understood that the embodiment analyzes the global image, extracts the edge contour of the bearing groove as the bearing contour, and takes the geometric center of the bearing contour as the analysis center. The global image is coordinate processed to establish a two-dimensional coordinate system, and the coordinates of the analysis center are determined as the analysis position of the bearing groove.

[0154] B2, performing binary processing on the global image, and obtaining the quantity of white pixel points at each analysis position as the analysis quantity.

[0155] It can be understood that the global image is binary processed in the prior art, the image is converted into a black and white image, the analysis position obtained based on step B1 is obtained, and the quantity of white pixel points at each analysis position is obtained as the analysis quantity.

[0156] B3, comparing the analysis quantity with a preset pixel point quantity range to obtain a target quantity of the target object at the corresponding analysis position, and taking an analysis position with a target quantity greater than or equal to 1 as a target position.

[0157] It can be understood that, through step B2, the white pixel point quantity (analysis quantity) at the analysis position is obtained. According to the corresponding actual load quantity at each analysis position, the corresponding preset pixel point quantity range in the quantity transmittance contrast table is obtained, and the analysis quantity is compared with the preset pixel point quantity range to obtain the target quantity of the target object at the corresponding analysis position.

[0158] B4, determining the target object based on the target position.

[0159] In some embodiments, (determining the target object based on the target position) in step A6 or B4 includes C1-C3:

[0160] C1, in response to the grabbing information, selecting a target position with an actual load quantity equal to a target quantity as a grabbing position, and selecting a target position with an actual load quantity greater than a target quantity as an adjustment position.

[0161] It can be understood that, after obtaining the target quantity of each target position, the target object needs to be sorted out. For example, when it is determined that 1 cocoon is a pupated cocoon, it can be directly selected; and when 2 cocoons are a pupated cocoon and an unpupated cocoon, they cannot be uniformly processed.

[0162] Further, the server, in response to the grabbing information, selects a target position with an actual load quantity equal to a target quantity as a grabbing position, indicating that the target object can be directly grabbed, and selects a target position with an actual load quantity greater than a target quantity as an adjustment position, indicating that the target object and the non-target object need to be sorted out.

[0163] C2, sending the coordinates corresponding to the grabbing position to the grabbing device to obtain the target object.

[0164] It can be understood that all the target objects are at the grabbing position, and the server converts the coordinates of the grabbing position into mechanical coordinates recognizable by the device and sends them to the grabbing device to grab the target object.

[0165] C3, obtaining the coordinates corresponding to the adjustment position and sending them to the adjustment end, and selecting the target object corresponding to the target quantity based on the adjustment end.

[0166] It can be understood that the adjustment position contains target objects and non-target objects, and the two are stacked and blocked with each other, and need to be intervened by the adjustment end. The server sends the coordinates of the adjustment position and the target quantity to the adjustment end, and the operating personnel or intelligent auxiliary equipment of the adjustment end selects the corresponding target object according to the target quantity.

[0167] Referring toFigure 6 is a structural schematic diagram of an automatic screening device based on image analysis provided by an embodiment of the present application, and the device comprises:

[0168] a distribution module, configured to determine a processing device of the screening object based on a bearing image of the bearing plate, wherein the processing device comprises a first processing device and a second processing device;

[0169] a first irradiation module, configured to, when the processing device is the first processing device, call the first lighting device to irradiate the screening object in each bearing groove on the bearing plate in sequence and perform image acquisition to obtain a local image;

[0170] a second irradiation module, configured to, when the processing device is the second processing device, call the second lighting device to irradiate the entire bearing plate and perform image acquisition to obtain a global image;

[0171] an analysis module, configured to perform image analysis on the local image or the global image to obtain a target object.

[0172] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An automated screening method based on image analysis, characterized in that, include: A processing device for determining screening items based on a carrier image of a carrier plate, the processing device comprising a first processing device and a second processing device, including: The bearing image of the bearing plate is obtained by the front image acquisition unit, and the bearing status of each bearing groove on the bearing plate is identified based on the bearing image. When it is determined that there are screening materials in the bearing tank, the bearing state of the corresponding bearing tank is marked as the actual load state, and the bearing tank in the actual load state is taken as the actual load tank. The number of actual load tanks is counted as the bearing quantity. The number of bearing grooves on the bearing plate is obtained as the full load quantity. The full load quantity is compared with the bearing quantity. When it is determined that the full load quantity is greater than the bearing quantity, the first processing device is selected as the processing device for the bearing plate. When the full load quantity is determined to be equal to the bearing capacity, the second processing device is selected as the processing device for the bearing plate. When the processing device is determined to be the first processing device, the first lighting equipment is activated to sequentially illuminate the screening materials in each carrier slot on the carrier plate and acquire images to obtain local images, including: The support plate is moved into the first processing device, and the support contour of each support groove is obtained based on the support image. The center of the support contour is determined as the support center. The image of the bearing is processed into coordinates to determine the coordinates of the bearing center as the bearing position of the bearing groove; The bearing position of each actual load slot is obtained as the actual load position. Based on the actual load position, the first lighting device above the bearing plate is called to illuminate each actual load slot in sequence, and the image of the receiving plate below the bearing plate is collected as a local image. When the processing device is determined to be the second processing device, the second lighting equipment is activated to illuminate the entire support plate and acquire images to obtain a global image, including: The carrier plate is moved into the second processing device, and the second lighting device located below the carrier plate is controlled to irradiate the screening material in the entire carrier plate through the transmission holes at the bottom of each carrier tank. A global image of the screened material on the carrier plate is acquired using a second image acquisition device located above the carrier plate. Image analysis is performed on the local or global image to obtain the target object, including: The number of screening materials in each load-bearing slot is determined based on the load image, and the actual load quantity is obtained. Based on the actual load quantity, obtain the preset pixel quantity range corresponding to the actual load quantity from the preset quantity transmittance lookup table; The target object is obtained by comparing a local or global image with a preset range of pixel counts.

2. The method according to claim 1, characterized in that, The process of sequentially illuminating each load slot using a first lighting device above the support plate based on the actual load position, and acquiring an image of the receiving plate below the support plate as a local image, includes: The first image acquisition device in the receiving area acquires the light spot mapped from the transmission hole at the bottom of the actual slot onto the receiving plate. The receiving area is the area between the receiving plate and the carrier plate, and the light spot is located in the receiving slot on the receiving plate that is symmetrical to the carrier slot. When it is determined that each of the first lighting devices is in its actual position, the first image acquisition device is activated to acquire the image of the light spot on the receiving board, thereby obtaining a local image of the filter material in the actual position.

3. The method according to claim 1, characterized in that, The process of comparing a local or global image with a preset range of pixel counts to obtain the target object includes: The receiving contour of the receiving slot is obtained from the local image, and the center of the receiving contour is determined as the receiving center. For a local image, a coordinate system is established with the same origin as the carrying image, and the coordinates of the receiving center are obtained as the receiving position of the receiving slot. Based on the actual location corresponding to the carrying image, the corresponding receiving location in the local image is determined as the detection location; The local image is binarized, and the number of white pixels at each detection location is used as the detection count. The number of targets at the corresponding actual location is obtained by comparing the number of detected objects with the preset range of the number of pixels, and the actual location with a target number greater than or equal to 1 is taken as the target location. The target object is determined based on its location.

4. The method according to claim 1, characterized in that, The process of comparing a local or global image with a preset range of pixel counts to obtain the target object includes: The bearing contours of each bearing groove are obtained based on the global image. The center of the bearing contour is determined as the analysis center. The global image is then processed into coordinates to determine the coordinates of the analysis center as the analysis position of the bearing groove. The global image is binarized, and the number of white pixels at each analysis location is used as the analysis count. The number of targets at the corresponding analysis position is obtained by comparing the number of analysis with the preset range of the number of pixels, and the analysis position with a target number greater than or equal to 1 is taken as the target position. The target object is determined based on its location.

5. The method according to claim 3 or 4, characterized in that, The method of determining the target object based on the target location also includes: In response to the crawling information, select the target location where the actual number of loads is equal to the target number as the crawling location, and select the target location where the actual number of loads is greater than the target number as the adjustment location; Send the coordinates corresponding to the grasping position to the grasping device to obtain the target object; The coordinates corresponding to the adjustment position are obtained and sent to the adjustment terminal, and the corresponding number of target objects are selected based on the adjustment terminal.

6. An automated screening device based on image analysis according to any one of claims 1-5, characterized in that, include: The allocation module is used to determine the screening device based on the carrying image of the carrier plate, the processing device including a first processing device and a second processing device; The first irradiation module is used to, when the processing device is determined to be the first processing device, invoke the first lighting device to irradiate the screening materials in each carrying groove on the carrying plate in turn and perform image acquisition to obtain local images. The second illumination module is used to, when the processing device is determined to be the second processing device, invoke the second lighting device to illuminate the entire carrier plate and perform image acquisition to obtain a global image. The analysis module is used to perform image analysis on the local or global image to obtain the target object.

Images