A feeding method and device for production line

Abstract

The present application discloses a feeding method and device for a production line, the method comprising: obtaining the working status of the production line, and determining the material information of the material to be processed transported by the material transport vehicle according to the working status; obtaining a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle; determining the position information corresponding to the material to be processed according to the material image; and controlling the material transport vehicle to grab the designated material that matches the material information among the materials to be processed according to the position information, and transporting the designated material to the production line through the material transport vehicle. Even if there are slight errors in the position and placement of the material, rapid identification and grabbing can be achieved. Moreover, grabbing and transporting the material to be processed based on the working status of the production line can be more in line with the actual production and processing status.

Description

Technical Field

[0001] The present application relates to the field of image data processing, and in particular to a material loading method and equipment for a production line. Background Art

[0002] With the development of technology, more and more automated equipment is appearing in the production and processing of products. For example, in the automated production process, self-driving material transport vehicles (such as automatic guided vehicles (AGVs)) transport materials stored in the loading area to the production line.

[0003] Normally, when a material transport vehicle loads materials in the loading area, it often uses a fixed method and amount to load the materials. After the materials are neatly placed in the loading area, the material transport vehicle goes to a fixed position, grabs a fixed amount of materials each time (usually the upper limit of the single delivery of the material transport vehicle), and then transports them to the production line through automatic navigation.

[0004] However, it still has the following problems:

[0005] 1. The material grabbing process relies too much on the setting of fixed positions and the placement of materials, which makes the material grabbing process too rigid and difficult to change in real time according to actual conditions. Once the material placement after unloading in the previous stage does not meet expectations, manual adjustment is required.

[0006] 2. The transportation quantity is not flexible enough and it is difficult to adjust in real time according to the actual situation. If the number of grabs is too large, the materials will be piled up on the production line, and if the number of grabs is too small, it will easily lead to low production efficiency. Summary of the Invention

[0007] In order to solve the above problems, the present application proposes a feeding method for a production line, comprising:

[0008] Determine that the automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line;

[0009] Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity;

[0010] Determining that the material transport vehicle has arrived at a preset position in the loading area, and acquiring a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle;

[0011] Determining position information corresponding to the material to be processed according to the material image;

[0012] According to the position information, the material transport vehicle is controlled to grab a designated material matching the material information from the material to be processed, and the designated material is transported to the production line by the material transport vehicle.

[0013] On the other hand, the present application also proposes a feeding device for a production line, comprising:

[0014] at least one processor; and,

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the following operations:

[0017] Determine that the automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line;

[0018] Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity;

[0019] Determining that the material transport vehicle has arrived at a preset position in the loading area, and acquiring a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle;

[0020] Determining position information corresponding to the material to be processed according to the material image;

[0021] According to the position information, the material transport vehicle is controlled to grab a designated material matching the material information from the material to be processed, and the designated material is transported to the production line by the material transport vehicle.

[0022] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0023] Determine that the automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line;

[0024] Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity;

[0025] Determining that the material transport vehicle has arrived at a preset position in the loading area, and acquiring a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle;

[0026] Determining position information corresponding to the material to be processed according to the material image;

[0027] According to the position information, the material transport vehicle is controlled to grab a designated material matching the material information from the material to be processed, and the designated material is transported to the production line by the material transport vehicle.

[0028] The feeding method for the production line proposed in this application can bring the following beneficial effects:

[0029] For most materials to be processed, image recognition can identify their location and allow them to be grabbed based on that information. This allows for rapid identification and grabbing, even with slight errors in the material's placement and placement. Furthermore, grabbing and transporting materials based on the production line's operating status better aligns with actual production and processing conditions, reducing material backlogs and inefficiencies caused by grabbing too many or too few items. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 Schematic diagram of the process of the loading method for the production line in the embodiment of the present application;

[0032] Figure 2 A schematic diagram of camera calibration in an embodiment of the present application;

[0033] Figure 3 This is a schematic diagram of image recognition by template matching in an embodiment of the present application;

[0034] Figure 4a and Figure 4b They are schematic diagrams of a material image and a template matching processed image under the template matching method in an embodiment of the present application;

[0035] Figure 5 This is a schematic diagram of the edge recognition results in an embodiment of the present application;

[0036] Figure 6 This is a schematic diagram of the loading equipment for the production line in an embodiment of the present application. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0039] like Figure 1 As shown, the embodiment of the present application provides a material loading method for a production line, comprising:

[0040] S101: Determine that an automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line.

[0041] The automated production system is typically located within a factory building, but in some cases, it can span multiple buildings or be deployed outdoors. Materials to be processed are transported from upstream nodes (e.g., raw material nodes, upstream processing nodes, etc.) and placed in a loading area.

[0042] The materials to be processed can include many types. For example, they can be divided into raw materials, semi-finished products, etc. according to the processing process, and can also be divided into food, electronic products, daily necessities, etc. according to the industry. The material arrangement method of different types of materials to be processed in the loading area is also different. For example, stable and non-fragile materials such as wooden boards and iron plates can be stacked, while fragile items such as glass can only be placed in one layer, or they can be placed in multiple layers through display racks. Products with painted surfaces need to be placed at intervals.

[0043] The material transport vehicle can be an automated guided vehicle (AGV), an industrial vehicle that can automatically or manually load materials, automatically drive along a set route or tow a loaded cart to a designated location, and then automatically or manually unload the materials. In this paper, the loading process is automated. The material transport vehicle is equipped with a gripping mechanism such as a robotic arm or a mechanical pallet, which grabs the materials to be processed and transports them to the storage area of the material transport vehicle.

[0044] S102: Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity.

[0045] When there are multiple production lines, the working status may include the type of material currently being processed by the production line. When the production line is a specified line and the materials to be processed stored in the loading area are also of a fixed type, the material type in the material information is a fixed specified type.

[0046] The working status also includes the production efficiency and material shortage for the material type within the production line. Production efficiency can be measured in minutes, indicating the number of materials processed per minute. The material shortage represents the amount of material currently required by the production line. This quantity is provided by the production line and can represent the shortage at the current moment or an estimated timeframe in the future. The material shortage can be calculated based on the maximum production efficiency.

[0047] S103: Determine that the material transport vehicle has arrived at a preset position in the loading area, and obtain a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle.

[0048] The preset position may be in the loading area or at the edge of the loading area. The preset position may be a preset fixed position or a preset qualified position (for example, the preset position may be a position near the material to be processed).

[0049] The image acquisition device is arranged on the material transport vehicle, and multiple image acquisition devices can be arranged at different positions to facilitate the acquisition of material images in a larger range.

[0050] Calibrate these multiple image acquisition devices in advance, for example, use Zhang Zhengyou calibration method, multiple image acquisition devices joint calibration, use edge scanning, and obtain the common field of view through feature point matching such as SIFT, such as Figure 2 As shown in the figure, the intersection of the acquisition areas of the two upper image acquisition devices is the public field of view. Then, the known internal calibration information is used to complete the solution of the coordinate system transformation relationship of multiple image acquisition devices to achieve the splicing of multiple images.

[0051] S104: Determine the position information corresponding to the material to be processed according to the material image.

[0052] By performing object recognition on the material image, the position information of the material to be processed can be identified. The representation of the position information may vary depending on the object recognition method.

[0053] Specifically, target recognition is performed by template matching, such as Figure 3 As shown in Figure 1, based on expert experience, a target template image with a size similar to the material to be processed in the image is created. The source image to be inspected (that is, the material image in this article) is traversed in a certain order (for example, from left to right, from top to bottom), and the pixel similarity of the overlapping image of the template image and the material image is calculated. The similarity calculated for each position is stored in the image matrix. In the image matrix, the whitest position represents the highest similarity, as shown in Figure 1. Figure 4a and Figure 4b As shown in the figure, in the template matching processed image, the position framed by the ellipse is likely to be the maximum value in the result image matrix, so this area is considered to be the result of target recognition, corresponding to the corresponding position in the material image.

[0054] Target recognition is performed through deep learning, using the Yolo series of deep learning detection algorithms to detect and identify materials to be processed in material images. Training is performed on captured high-resolution images to extract and analyze the device's features. During model training, dense connections using dilated convolutions with varying dilation coefficients are used to balance the receptive field, while embedding multi-layer output fusion to further improve the model's detection accuracy. In actual inspection operations, the trained model's understanding of samples, combined with extracted local details and global contextual information, accurately identifies the location of target engine components.

[0055] Target recognition is performed through edge detection, and the material image is preprocessed to produce a preprocessed image. This preprocessing process may include grayscale processing, image noise reduction, and image enhancement. For each pixel in the preprocessed image, the corresponding image intensity function is derived. The corresponding two-dimensional vector is determined based on the derivative result and used as the gradient vector.

[0056] The image intensity function is derived by combining the image intensities of each pixel and represents the image intensity of each pixel in the preprocessed image. Derivatives can be performed using forward, backward, and central differencing methods, and second-order derivatives are defined. The derivative results reflect the rate of decrease of the image intensity function; the direction of the fastest decrease is the direction of the gradient. The obtained derivative results define the two-dimensional vector of the image intensity function at each pixel. For example, by performing differential derivatives in the x and y directions, combining them and obtaining the transposed matrix, a two-dimensional column vector is obtained.

[0057] Generate a gradient image based on the amplitude of the gradient vector of all pixels, construct an operator (such as Robert operator, Sobel operator, Prewitt operator, Kirsch operator, Gauss-Laplace operator, Canny operator, etc.), and perform cross-correlation operation between the operator and the gradient image to obtain the corresponding edge pixels in the material image, thereby completing edge recognition. Figure 5 As shown, the horizontal line near the middle of the image represents the edge of the material to be identified.

[0058] Due to the complex on-site environment, the edge pixels ultimately identified may not only be those of the material to be identified, but may also be those of other objects in the on-site environment (e.g., the support frame on which the material is placed, lines drawn on the ground, etc.). Therefore, the edge pixels present in the pre-set image area are regarded as the edges of the material to be processed. The preset position for the material transport vehicle to arrive at and the placement range of the material to be processed in the loading area are pre-set, and the shooting range of the image acquisition device is also known. Therefore, it is possible to determine which areas of the material image the material to be identified appears in. These areas are used as image areas, and only the edge pixels that appear in the image area are used as position information. Edge pixels that appear outside the image area are filtered out.

[0059] For most materials to be processed, simply capturing these edge pixels is sufficient to grasp the material. However, for some fragile materials, lifting and other operations are required during the grasping process, requiring additional positional information. Material transport vehicles may have limited shooting angles, and edge pixels appearing in the image area are used as lateral positional information. The material dimensions and layout are already captured in advance. The dimensional information includes length, width, and height. The material layout can be used to determine how the material is placed. Combined with the known lateral positional information, the longitudinal and height positional information of the material to be processed can be determined.

[0060] S105: According to the position information, control the material transport vehicle to grab a designated material matching the material information from the materials to be processed, and transport the designated material to the production line via the material transport vehicle.

[0061] The designated material may include one or more pieces. When the material types include multiple types, the designated materials to be grabbed this time may also include multiple types. The grabbing process may be divided into multiple grabbing times, and the quantity to be grabbed each time is preset.

[0062] For most materials to be processed, image recognition can identify their location and allow them to be grabbed based on that information. This allows for rapid identification and grabbing, even with slight errors in the material's placement and placement. Furthermore, grabbing and transporting materials based on the production line's operating status better aligns with actual production and processing conditions, reducing material backlogs and inefficiencies caused by grabbing too many or too few items.

[0063] In one embodiment, when recognizing an image, the material to be processed can be obtained through methods such as template matching, edge recognition, and deep learning. Compared to methods such as template matching and deep learning, edge recognition relies more on pixel variations in the image because template matching can use a target template image with a larger range than the edge, and deep learning can consider more dimensions. Therefore, any abnormalities in the external environment (e.g., strong lighting) can easily affect the edge recognition results.

[0064] Based on this, during edge recognition, the material image is preprocessed by grayscale processing to obtain a grayscale image. If the light reflectance coefficient of the surface of the material to be processed is determined to be higher than a preset coefficient, the light source present within the three-dimensional scene model corresponding to the loading area is determined. The surface materials of various types of materials to be processed and the light reflectance coefficients of the surface materials can be pre-acquired and stored, and the existing light sources can include internal light sources and external light sources.

[0065] The intensity of internal light sources is known and easily accessible, while the intensity of external light sources can be detected by light sensors installed near doorways and windows, for example. If the total light intensity of the light sources at the current moment exceeds the preset intensity, it indicates that the light is too strong and can easily affect edge recognition of the material being processed.

[0066] At this time, for each light source, determine the lighting effect of the light source on the material to be processed. The lighting effect refers to the reflected light obtained by irradiating the material to be processed with the light source as the incident light. After superimposing the lighting effects corresponding to all light sources, the total light intensity received by the surface of the material to be processed is obtained.

[0067] Illumination methods include direct light and / or indirect light. Direct light refers to the reflected light generated by the light source directly illuminating the surface of the material to be processed, while indirect light refers to the reflected light generated by the light source after irradiating the surfaces of other objects in the 3D scene model, reflecting one or more times, and then reflecting back to the surface of the material to be processed. The more reflections, the lower the corresponding lighting impact of the indirect light. The higher the light reflection coefficient, the more sensitive it is to the influence of lighting. Therefore, the lighting impact needs to include more reflections corresponding to indirect light, that is, the lighting impact caused by more reflections needs to be considered.

[0068] According to the influence of lighting, the grayscale value of the corresponding pixel of the grayscale image is reduced, and the grayscale image after the grayscale value is reduced is used as a preprocessing image to reduce the negative impact of lighting on edge recognition.

[0069] Furthermore, when determining the illumination effect of the light source on the material to be processed, the illumination effect of the light source on the material to be processed is obtained by formula 1, wherein formula 1 is: L = F + KE + K 2 E+K 3 E+…+K n E, L are the total reflected light intensity of the light source, which is used to represent the lighting effect. F is the self-luminous intensity (when the material to be processed emits light, its corresponding self-luminous intensity can be obtained. If it does not emit light, there is no need to consider the impact caused by this part, which will not be described in detail below). n is the number of reflections, K n E is the indirect light intensity when the light acts on the material to be processed after n reflections. When n=1, KE is the direct light intensity. At this time, the higher the light reflection coefficient, the greater the number of reflections n corresponding to the indirect light that needs to be included in the lighting effect, that is, more indirect light intensity K needs to be included. n E.

[0070] For direct light intensity and indirect light intensity, the reflected light intensity of each reflection process is obtained by formula 2, where formula 2 is: L r (x,ω r )=fL i (x,ω i )cosθ i .

[0071] Among them, ω r is the viewing direction, ω i is the incident direction, x is the illumination position of this light, L r (x,ω r ) is the intensity of the reflected light during this reflection process, L i (x,ω i ) is the incident light intensity during this reflection process, for direct light intensity, θi is the offset angle of the light source relative to the x position, for indirect light intensity, θ i is the offset angle of the previous reflection point relative to the x position, and f is the bidirectional reflectance distribution function.

[0072] The bidirectional reflectance distribution function (BRDF) can be considered a quantitative unification of materials, using a physically based rendering (PBR) material model. It is derived through experimental measurements, real-world statistics, and the application of physical formulas. It represents the attenuation of light after rebound, thereby inferring the intensity of reflected light.

[0073] Thus, when considering direct or indirect light, after the light is emitted from the light source, as it reflects off each surface in turn, the incident and reflected light intensity of each reflection is calculated according to Formula 2. This reflected light intensity is then used as the incident light intensity for the next reflection, and this calculation is completed until the light reaches the surface of the material to be processed. The final calculation results in the corresponding lighting impact. Based on the set number of reflections, n, only light that reaches the surface of the material to be processed within n reflections is considered, and the total lighting impact of the light source is calculated according to Formula 1. By adding up the lighting impacts of all light sources, the final lighting impact corresponding to the material to be processed can be obtained.

[0074] Furthermore, the actual amount of light emitted by each light source is difficult to count, so for each light source, the illumination emitted by the light source is represented as a preset number of evenly distributed rays, thereby quantifying the illumination and dividing the surface of the material to be processed into multiple sub-areas, and the sum of the areas of all sub-areas is the total surface of the material to be processed.

[0075] For each sub-region, determine whether each light ray can reach the sub-region through m reflection processes, where m≤n, the number of n is determined according to the light reflection coefficient, and both m and n are positive integers.

[0076] If it can reach the target, it indicates that the light may have an impact on the illumination of the material to be processed. According to Formula 2, after m calculations and the light path of the light, the reflected light intensity of the sub-area corresponding to the light source is calculated. For each calculation, if the current light directly affects the material to be processed, the reflected light intensity during this reflection process is at least part of the illumination impact. If the reflected light intensity does not directly affect the material to be processed, but acts on the surface of other objects in the 3D scene model, the reflected light intensity during this reflection process is the incident light intensity during the next reflection process.

[0077] According to Formula 1, all illumination effects within the subregion are summed to obtain the illumination effect corresponding to that subregion. To eliminate the impact of illumination effects on grayscale values, the grayscale values of the pixels within each subregion in the grayscale image are reduced based on the illumination effect corresponding to that subregion. The degree of reduction is positively correlated with the magnitude of the illumination effect; that is, the greater the illumination effect within the subregion, the greater the grayscale value reduction. This segmentation of the material image allows for more precise elimination of the negative effects of illumination effects.

[0078] The above-mentioned solutions can be used to determine the effects of lighting on each frame of a material image. During the material handling process, the image acquisition device operates in real time, capturing multiple frames. Using the above-mentioned solutions to determine lighting effects on every frame would easily require excessive computing power.

[0079] Based on this, the three-dimensional scene model is discretized into multiple facets, and at the moment corresponding to the current frame (usually the first frame), for each facet, the lighting effect received by the facet is obtained by formula 1, and the lighting effects corresponding to all facets are stored in the surface cache. It should be noted that when discretizing facets, not only the materials to be processed are discretized, but all objects in the three-dimensional scene model are discretized, and the current frame is used to represent the corresponding moment, not to represent the material image of the current frame. The surface cache can be understood as a separate area in the hard disk for storing the corresponding lighting effects.

[0080] For a scene or any surface element, the total energy at a certain moment is constant. Therefore, for any surface element, the energy radiated by it is equal to the sum of the energies radiated to it by other surfaces elements. For different moments corresponding to different frames, due to the short time interval between the two moments, the energies of two adjacent frames are approximately considered equal. In this case, a multiplexing method can be adopted. At the moment corresponding to the next frame, the illumination influence corresponding to each surface element stored in the surface cache (that is, the illumination influence received by each surface element at the moment corresponding to the previous frame) is used as the illumination energy radiated outward by the surface element. According to Formula 3, the illumination influence received by all surfaces at the moment corresponding to the next frame is obtained. The illumination influence stored in the surface cache is updated based on the illumination influence received at the moment corresponding to the next frame. This cycle repeats, storing the illumination influence received at the previous frame moment in the surface cache and using it as the illumination energy radiated outward at the next frame moment. There is no need to solve Formulas 1 and 2 for each frame, reducing the computing power required.

[0081] Among them, formula three is: Among them, E j is the self-luminous intensity of the jth surface element, B jis the illumination effect received by the jth surface element, B i is the light energy radiated outward by the i-th surface element, ρ j is the light reflection coefficient of the jth surface element, F ij is the shape factor corresponding to the i-th surface element and the j-th surface element, which is used to describe the corresponding relationship between the shapes of the surface elements in space.

[0082] The lighting effects corresponding to each sub-area have been determined above. Lighting effects often make the surface of the material to be processed whiter, and the corresponding grayscale value will also increase accordingly. Therefore, it is necessary to reduce the grayscale value to complete the correction of the lighting effects.

[0083] Specifically, the lighting impact of each sub-area is determined, and the first sub-area whose lighting impact is higher than a preset level is screened out. Only the sub-area with a higher lighting impact needs to be corrected, and other sub-areas with a lower impact (lower than the preset level) do not need to be corrected.

[0084] For each first subregion, determine the corresponding reduction ratio (greater than 0 and less than 1) for the illumination impact of the first subregion. The higher the illumination impact, the greater the grayscale value that needs to be reduced, so there is a positive correlation between illumination impact and reduction ratio. Then, determine the second subregion closest to the first subregion among all subregions that does not belong to the first subregion.

[0085] The grayscale values of the pixels in the first sub-region are reduced according to the reduction ratio. For example, if the original grayscale value is 0 and the reduction ratio determined according to the mapping relationship is 10%, the grayscale value after reduction is 0*(1-10%).

[0086] If the grayscale value of the first sub-region after the reduction is lower than the preset minimum grayscale value, the preset minimum grayscale value is used as the grayscale value of the first sub-region after the reduction to prevent image distortion caused by excessive correction. The preset minimum grayscale value is the average of the grayscale values of the second sub-region, which can make the corrected image smoother.

[0087] Furthermore, considering that the correction process may lose information about edge pixels, the illumination effects of each first subregion within the range corresponding to the first subregion are sequentially obtained, row or column by row, and a corresponding illumination intensity curve is generated based on this illumination effect. By taking a derivative of the illumination intensity curve, the rate of change of illumination intensity within each first subregion can be determined.

[0088] Generally speaking, if the surface material of the material to be processed does not change, the change rate should be almost the same. Therefore, if it is determined according to the derivation result that the light intensity change rate of the third sub-area (the third sub-area belongs to the first sub-area) has a sudden change (a mutation means that in this unit, the difference between the light intensity change rate of the third sub-area and the change rate of other adjacent sub-areas is higher than the preset difference), it means that the material may change. At this time, reducing the reduction ratio corresponding to the third sub-area (for example, reducing it by half) can prevent the loss of edge information caused by grayscale correction.

[0089] In one embodiment, when confirming the grab quantity of the material to be processed, the gap material type corresponding to the current material gap and the first gap material quantity are determined according to the working status. The gap material refers to the material currently required on the production line.

[0090] Determine the transport time of the material transport vehicle from the loading area to the production line, and determine the current production efficiency of the production line based on the working status. Based on the transport time and production efficiency, determine the corresponding second gap material quantity. The longer the transport time and the higher the production efficiency, the larger the second gap material quantity. This process can be pre-determined through historical data or experiments to obtain the corresponding mapping relationship and calculation formula.

[0091] At this time, the corresponding material type can be obtained according to the gap material type, and the corresponding material quantity can be obtained according to the first gap material quantity and the second gap material quantity.

[0092] Specifically, the undetermined quantity is obtained according to Formula 4, where Formula 4 is: u =α*(q1+q2), where q u is the undetermined quantity, q1 is the quantity of the first gap material, q2 is the quantity of the second gap material, and a is the type factor, which is set based on different material types. The higher the breakability of the material type, the higher the material value, and the lower the type factor. This relationship can be set manually in advance.

[0093] If the pending quantity is greater than or equal to the material transport vehicle's upper limit, the upper limit is used as the corresponding material quantity. If the pending quantity is less than the material transport vehicle's upper limit, the system determines whether the pending quantity is less than the preset minimum quantity. If the pending quantity is less than or equal to the preset minimum quantity, the preset minimum quantity is used as the corresponding material quantity. If the pending quantity is greater than the preset minimum quantity and less than the upper limit, the pending quantity is used as the corresponding material quantity.

[0094] like Figure 6 As shown, the embodiment of the present application also proposes a feeding device for a production line, including:

[0095] at least one processor; and,

[0096] a memory communicatively connected to the at least one processor; wherein,

[0097] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the following operations:

[0098] Determine that the automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line;

[0099] Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity;

[0100] Determining that the material transport vehicle has arrived at a preset position in the loading area, and acquiring a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle;

[0101] Determining position information corresponding to the material to be processed according to the material image;

[0102] According to the position information, the material transport vehicle is controlled to grab a designated material matching the material information from the material to be processed, and the designated material is transported to the production line by the material transport vehicle.

[0103] The present application also provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0104] Determine that the automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line;

[0105] Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity;

[0106] Determining that the material transport vehicle has arrived at a preset position in the loading area, and acquiring a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle;

[0107] Determining position information corresponding to the material to be processed according to the material image;

[0108] According to the position information, the material transport vehicle is controlled to grab a designated material matching the material information from the material to be processed, and the designated material is transported to the production line by the material transport vehicle.

[0109] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device and medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0110] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0111] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0116] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0117] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0119] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A feeding method for a production line, characterized in that: include: Determine that the automatic production system includes: a loading area for storing materials to be processed, a production line for processing the materials to be processed, and a material transport vehicle for transporting the materials to be processed from the loading area to the production line; Acquire the working status of the production line, and determine the material information of the material to be processed transported by the material transport vehicle according to the working status, wherein the material information includes the material type and the material quantity; Determining that the material transport vehicle has arrived at a preset position in the loading area, and acquiring a material image containing the material to be processed by an image acquisition device provided on the material transport vehicle; According to the material image, the position information corresponding to the material to be processed is determined, specifically including: grayscale processing of the material image to obtain a grayscale image; if it is determined that the light reflection coefficient of the surface of the material to be processed is higher than the preset coefficient, the light source existing in the three-dimensional scene model corresponding to the loading area is determined; if the total light intensity of the light source at the current moment exceeds the preset intensity, then for each light source, the light impact of the light source on the material to be processed is determined, wherein the light impact refers to the reflected light obtained by irradiating the material to be processed with the light source as the incident light, and the irradiation method includes direct light and / or indirect light. The higher the light reflection coefficient, the more reflections corresponding to the indirect light need to be included in the light impact; the light impact also includes self-luminous intensity. When the material to be processed itself emits light, its corresponding self-luminous intensity is obtained. If it does not emit light, the self-luminous intensity does not need to be considered; according to the light impact, the grayscale image is reduced. The grayscale value of the corresponding pixel is obtained, and the grayscale image after reducing the grayscale value is used as the preprocessed image; for each pixel in the preprocessed image, the corresponding image intensity function is derived, and the corresponding two-dimensional vector is determined according to the derivative result, so as to use the two-dimensional vector as the gradient vector; a gradient image is generated according to the amplitude of the gradient vector of all pixels; an operator is constructed, and a cross-correlation operation is performed between the operator and the gradient image to obtain the corresponding edge pixels in the material image; the edge pixels existing in the preset image area are used as the lateral position information of the material to be processed, and the image area is determined according to the shooting range of the image acquisition device at the preset position and the placement range of the material to be processed in the loading area; the material size information and material arrangement method also included in the material information are determined, and the longitudinal position information and height position information of the material to be processed are determined according to the material size information and the material arrangement method; According to the position information, the material transport vehicle is controlled to grab a designated material matching the material information from the material to be processed, and the designated material is transported to the production line by the material transport vehicle; For each light source, determining the lighting effect of the light source on the material to be processed includes: The lighting effect of the light source on the processed material is obtained by formula 1, wherein the formula 1 is: L=F+KE+K 2 E+K 3 E+…+K n E, L is the total reflected light intensity of the light source, used to represent the lighting effect, F is the self-luminous intensity, n is the number of reflections, K n E is the indirect light intensity when the light acts on the material to be processed after n reflections. When n=1, KE is the direct light intensity. The higher the light reflection coefficient, the greater the number of reflections n corresponding to the indirect light that needs to be included in the lighting effect. For direct light intensity and indirect light intensity, the reflected light intensity of each reflection process is obtained by formula 2, wherein the formula 2 is: L r (x,ω r )=fL i (x,ω i )cosθ i ; Among them, ω r is the viewing direction, ω i is the incident direction, x is the illumination position of this light, L r (x,ω r ) is the intensity of the reflected light during this reflection process, L i (x,ω i ) is the incident light intensity during this reflection process, for direct light intensity, θ i is the offset angle of the light source relative to the x position, for indirect light intensity, θ i is the offset angle of the previous reflection point relative to the x position, and f is the bidirectional reflectance distribution function; Discretizing the three-dimensional scene model into a plurality of facets; At the moment corresponding to the current frame, for each surface element, the illumination effect received by the surface element is obtained by using Formula 1, and the illumination effects corresponding to all surface elements are stored in the surface cache; At a time corresponding to the next frame, using the illumination influence corresponding to each bin stored in the surface cache as the illumination energy radiated outward by the bin, and obtaining the illumination influence received by all bins at the time corresponding to the next frame according to Formula 3, and updating the illumination influence stored in the surface cache according to the illumination influence received at the time corresponding to the next frame; Wherein, the formula three is: Among them, E j is the self-luminous intensity of the jth surface element, B j is the illumination effect received by the jth surface element, B i is the light energy radiated outward by the i-th surface element, ρ j is the light reflection coefficient of the jth surface element, F ij is the shape factor corresponding to the i-th bin and the j-th bin.

2. The method according to claim 1, characterized in that For each light source, determining the lighting effect of the light source on the material to be processed includes: For each light source, the illumination emitted by the light source is represented as a preset number of uniformly distributed rays, and the surface of the material to be processed is divided into a plurality of sub-regions, and the sum of the areas of all sub-regions is the total surface of the material to be processed; For each sub-region, determining whether each light ray can reach the sub-region through m reflection processes, where m≤n, and the number n is determined according to the light reflection coefficient; If it can reach the target, then the intensity of the reflected light corresponding to the light source of the sub-area is calculated m times according to Formula 2 and the optical path of the light. For each calculation, if the current illumination directly acts on the material to be processed, the intensity of the reflected light during the current reflection process is at least part of the illumination effect. If the reflected light intensity does not directly act on the material to be processed, the intensity of the reflected light during the current reflection process is the incident light intensity during the next reflection process. According to formula 1, all the lighting effects in the sub-area are summed to obtain the lighting effect corresponding to the sub-area; According to the illumination influence, reducing the grayscale value of the corresponding pixel of the grayscale image specifically includes: According to the illumination influence corresponding to each sub-region, the grayscale values of the pixels contained in the sub-region in the grayscale image are reduced, wherein the reduction degree is positively correlated with the value of the illumination influence.

3. The method according to claim 2, characterized in that According to the illumination influence corresponding to each sub-region, the grayscale value of the pixels in the sub-region in the grayscale image is reduced, specifically including: Determine the lighting impact of each sub-region, and select a first sub-region having a lighting impact higher than a preset level; For each first subregion, determine a reduction ratio corresponding to the illumination impact of the first subregion, and among all subregions, determine a second subregion that is closest to the first subregion and does not belong to the first subregion, wherein the illumination impact and the reduction ratio are positively correlated; reducing the grayscale values of the pixels within the first sub-region according to the reduction ratio, wherein if the grayscale value of the first sub-region after the reduction is lower than a preset minimum grayscale value, using the preset minimum grayscale value as the grayscale value after the reduction of the first sub-region, and the preset minimum grayscale value is the average of the grayscale values of the second sub-region; In the range corresponding to the first sub-area, sequentially obtaining the illumination influence corresponding to each first sub-area in the unit in rows or columns, and generating a illumination intensity curve corresponding to the unit by fitting according to the illumination influence; The light intensity curve is differentiated once. If it is determined according to the differentiation result that the light intensity change rate of the third sub-region has a sudden change, then the reduction ratio corresponding to the third sub-region is reduced. The third sub-region belongs to the first sub-region. The sudden change means that in this unit, the difference between the light intensity change rate of the third sub-region and the change rate of other adjacent sub-regions is higher than the preset difference.

4. The method according to claim 1, wherein Determining material information of the material to be processed transported by the material transport vehicle according to the working status specifically includes: According to the working status, determine the gap material type and the first gap material quantity corresponding to the current material gap; Determining the transportation time of the material transport vehicle from the loading area to the production line, and determining the current production efficiency of the production line based on the working status; Determine the corresponding second gap material quantity according to the transportation time and the production efficiency; The corresponding material type is obtained according to the gap material type, and the corresponding material quantity is obtained according to the first gap material quantity and the second gap material quantity.

5. The method according to claim 4, characterized in that Obtaining corresponding material quantities according to the first material shortfall quantity and the second material shortfall quantity specifically includes: The undetermined quantity is obtained according to Formula 4, wherein the Formula 4 is: q u =α*(q1+q2), where q u is the undetermined quantity, q1 is the quantity of the first gap material, q2 is the quantity of the second gap material, and α is the type factor, which is set based on different material types. The higher the breakability of the material type and the higher the material value, the lower the type factor; If the undetermined quantity is greater than or equal to the upper limit of the material transport vehicle, the upper limit is used as the corresponding material quantity; If the pending quantity is lower than the upper limit of the material transport vehicle, determining whether the pending quantity is lower than a preset minimum quantity; If the to-be-determined quantity is lower than or equal to the preset minimum quantity, the preset minimum quantity will be used as the corresponding material quantity; If the pending quantity is higher than the preset minimum quantity and lower than the transportation upper limit, the pending quantity will be used as the corresponding material quantity.

6. A feeding device for a production line, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the loading method for the production line as described in claim 1.

Images