Production detection method, device, electronic equipment and storage medium

Abstract

The present application relates to the field of automobile parts production lines, and in particular to a production inspection method, device, electronic device and storage medium, which are applied to a feeding system of at least two tooling trays, the method comprising obtaining an image to be inspected collected for each tooling tray, each image to be inspected including all component information in the corresponding tooling tray; performing target recognition on each image to be inspected to obtain the actual model of each component in each tooling tray; judging whether there are non-standard components in each tooling tray based on the actual model of each component in each tooling tray and the standard model corresponding to each tooling tray, the actual model of the non-standard component does not match the standard model, and each tooling tray is preset with a standard model for the component; if none of them exist, controlling the manipulator to grab the components in each tooling tray to a preset position. The present application can improve the yield rate of assembly of lower-level components.

Description

Technical Field

[0001] The present application relates to the field of automobile parts production lines, and in particular to a production detection method, device, electronic equipment and storage medium. Background Art

[0002] Product production in a factory sometimes requires the assembly of multiple lower-level parts. For example, on the assembly line of magnetic tubes and coils in the related technology, workers need to manually place the magnetic tubes and coils in the corresponding tooling pallets respectively. The pallets are moved to the assembly station via a conveyor belt and then grabbed by a robot for assembly.

[0003] Only the lower-level components with corresponding sizes can be assembled into qualified products. In fact, since the magnetic tube and coil have multiple sizes, workers will mix the lower-level components of different sizes when placing the lower-level components, resulting in lower-level components of different sizes in one tooling tray. In the subsequent assembly process, this will lead to assembly failure or assembly of unqualified products, resulting in a low yield rate of the production line. Summary of the invention

[0004] In order to improve the yield rate of assembly of lower-level components, the present application provides, in particular, a production detection method, device, electronic equipment and system.

[0005] In the first aspect, the present application provides a production detection method, which adopts the following technical solution:

[0006] A production detection method, applied to a feeding system of at least two tooling pallets, comprises:

[0007] Acquire an image to be inspected collected for each tooling tray, each of the images to be inspected including all component information in the corresponding tooling tray;

[0008] Performing target recognition on each of the images to be inspected to obtain the actual model of each component in each tooling tray;

[0009] Based on the actual model of each component in each tooling tray and the standard model corresponding to each tooling tray, it is determined whether there is a non-standard component in each tooling tray, the actual model of the non-standard component does not match the standard model, and each tooling tray is preset with a standard model for the component;

[0010] If none of them exist, the robot is controlled to grab the components in each tooling tray to the preset position.

[0011] By adopting the above technical solution, during the component conveying stage, images of all components in each tooling tray are obtained, and the model information of each component in each tray is obtained by target recognition. Then, the preset standard model of each tooling tray is used to determine whether there are non-standard components in each tooling tray. When there are no non-standard components in all tooling trays, the robot is controlled to grab the components to the preset position for assembly, thereby improving the yield rate of the assembled finished products.

[0012] In a possible implementation, before controlling the robot to grab the components in each tooling tray to a preset position, for any image to be inspected, the method further includes:

[0013] Acquire size information and angle information of each component in the image to be inspected;

[0014] Determine the actual size of each component in the image to be inspected based on the mapping relationship between the size information and the actual size of each component in the image to be inspected;

[0015] Determining the actual angle of each component in the image to be inspected based on the mapping relationship between the angle information of each component in the image to be inspected and the actual angle;

[0016] Determine whether the actual size of each component in the image to be inspected is within a first preset interval, and determine whether the actual angle of each component in the image to be inspected is within a second preset interval;

[0017] If both are located at the preset position, the robot arm is controlled to grab the components in the tooling tray to the preset position.

[0018] By adopting the above technical solution, it is determined whether the component meets the grasping standard of the robot through the actual size of the component and the first preset interval, and then it is determined whether the angle of the component meets the grasping standard of the robot through the actual angle of the component and the second preset interval. The robot will only grasp the component when the actual size and actual angle of the component both meet the grasping standard of the robot, thereby improving the success rate of the robot grasping the component.

[0019] In one possible implementation, the method further includes:

[0020] Controlling the preset device to scan the preset code printed on each component in each tooling tray to record the preset code on each component, wherein the preset code is associated with the production batch information of the corresponding component;

[0021] The codes corresponding to the components are parsed to obtain the production batches corresponding to the components.

[0022] By adopting the above technical solution, the preset code of each component is recorded by scanning, and the production batch information of each component can be obtained after parsing the preset code, so as to facilitate the subsequent tracing of the production batch of the problematic component when a problematic component occurs.

[0023] In a possible implementation, for any image to be inspected, the method further includes:

[0024] Performing image segmentation on the image to be inspected to obtain target areas corresponding to the components in the image to be inspected;

[0025] Extracting features from each of the target areas to obtain feature information corresponding to each element, wherein the feature information includes at least color features and texture features;

[0026] Determining the material category of each component based on the feature information corresponding to each component and the preset mapping relationship between the feature information and the material category;

[0027] Determining whether the material categories of all the components are preset standard categories;

[0028] If not, output category warning information.

[0029] By adopting the above technical solution, components of the same size may also be made of different materials. The material category of the component is obtained through the characteristic information of the component. When at least one non-standard component appears, a category warning message is output to facilitate users to discover it in time, thereby reducing the probability of unqualified finished products.

[0030] In a possible implementation, the controlling the manipulator to grab a preset position of a component in each tooling tray includes:

[0031] Obtaining the image coordinates of each component in each of the images to be inspected in the pre-inspection image;

[0032] Determine the actual coordinates of each component in each of the preview images based on the image coordinates of each component in the preview image and the conversion relationship between the image coordinates and the actual coordinates of the component, wherein the actual coordinates are the coordinates of the target component in the corresponding tooling tray;

[0033] Get the assembly coordinates of the preset position;

[0034] Based on the actual coordinates of each component and the assembly coordinates, the manipulator is controlled to grab the components in each tooling tray to a preset position.

[0035] By adopting the above technical solution, the robot needs to accurately grasp the components in the tooling tray. It needs to know the position information of the grasped components, that is, the actual coordinates of the components. By determining the coordinates of the components in the image to be inspected, the actual coordinates of the components on the tooling tray are obtained based on the conversion relationship.

[0036] In a possible implementation, the acquiring of the image coordinates of each component in each of the preview images in the preview image, for any of the preview images, includes:

[0037] Establishing an image coordinate system based on the preview image;

[0038] The coordinates of each component in the preview image in the preview image coordinate system are determined as image coordinates corresponding to each component in the preview image.

[0039] In one possible implementation, the method further includes:

[0040] Receive assembly completion instructions;

[0041] Acquire assembly information of each finished product, where the finished product is assembled from at least one component in each tooling tray, and the assembly information at least includes assembly time and assembly pressure of the finished product;

[0042] Obtain the inspection information of each finished product by the preset inspection equipment;

[0043] Traceability information is generated and associated with the preset codes of the components of the finished assembly, wherein the traceability information includes at least the assembly information and the detection information as well as the material category, the production batch and the actual model of the components of the finished assembly.

[0044] By adopting the above technical solution, the generated traceability information is associated with the preset code on each component that constitutes the finished product. This can effectively utilize the role of the damaged code and save time. By scanning the preset code of any component on the finished product and parsing it, the traceability information of the finished product in the production process can be obtained.

[0045] In a second aspect, the present application provides a production detection device, which adopts the following technical solution:

[0046] A production detection device, comprising

[0047] An image acquisition module for inspection, used to acquire an image for inspection collected for each tooling tray, each of which includes information of all components in the corresponding tooling tray;

[0048] A target recognition module is used to perform target recognition on each of the images to be inspected to obtain the actual model of each component in each tooling tray;

[0049] A first judgment module is used to judge whether there is a non-standard component in each tooling tray based on the actual model of each component in each tooling tray and the standard model corresponding to each tooling tray, and the actual model of the non-standard component does not match the standard model, and each tooling tray is preset with a standard model for the component;

[0050] The control module is used to control the robot to grab the components in each tooling tray to a preset position.

[0051] By adopting the above technical solution, during the component conveying stage, the device can obtain the image to be inspected including all components in each tooling tray, obtain the model information of each component in each tray by means of target recognition, and then judge whether there are non-standard components in each tooling tray according to the preset standard model of each tooling tray. When there are no non-standard components in all tooling trays, the robot arm is controlled to grab the components to the preset position for assembly, thereby improving the yield rate of the assembled finished products.

[0052] In a possible implementation, the production detection device further includes:

[0053] A first acquisition module, used to acquire size information and angle information of each component in the image to be inspected;

[0054] An actual size determination module, used to determine the actual size of each component in the image to be inspected based on the mapping relationship between the size information of each component in the image to be inspected and the actual size;

[0055] An actual angle determination module, used to determine the actual angle of each component in the image to be inspected based on the mapping relationship between the angle information of each component in the image to be inspected and the actual angle;

[0056] The second judgment module is used to judge whether the actual size of each component in the image to be inspected is within the first preset interval, and judge whether the actual angle of each component in the image to be inspected is within the second preset interval.

[0057] In a possible implementation, the production detection device further includes:

[0058] Controlling the preset device to scan the preset code printed on each component in each tooling tray to record the preset code on each component, wherein the preset code is associated with the production batch information of the corresponding component;

[0059] The codes corresponding to the components are parsed to obtain the production batches corresponding to the components.

[0060] In one possible implementation, the production device further includes:

[0061] An image segmentation module is used to perform image segmentation on the image to be inspected to obtain target areas corresponding to each component in the image to be inspected;

[0062] A feature extraction module, used to extract features from each target area to obtain feature information corresponding to each element, wherein the feature information includes at least color features and texture features;

[0063] A material category determination module, used to determine the material category of each component based on the feature information corresponding to each component and a preset mapping relationship between the feature information and the material category;

[0064] A third judgment module is used to judge whether the material categories of all the components are preset standard categories;

[0065] The warning information output module is used to output category warning information.

[0066] In a possible implementation, when the control module controls the manipulator to grab the preset position of the component in each tooling tray, it is specifically used to:

[0067] Obtaining the image coordinates of each component in each of the images to be inspected in the pre-inspection image;

[0068] Determine the actual coordinates of each component in each of the preview images based on the image coordinates of each component in the preview image and the conversion relationship between the image coordinates and the actual coordinates of the component, wherein the actual coordinates are the coordinates of the target component in the corresponding tooling tray;

[0069] Get the assembly coordinates of the preset position;

[0070] Based on the actual coordinates of each component and the assembly coordinates, the manipulator is controlled to grab the components in each tooling tray to a preset position.

[0071] In a possible implementation, when the control module obtains the image coordinates of each component in each of the preview images in the preview image, for any of the preview images, it is specifically configured to:

[0072] Establishing an image coordinate system based on the preview image;

[0073] The coordinates of each component in the preview image in the preview image coordinate system are determined as image coordinates corresponding to each component in the preview image.

[0074] In a possible implementation, the production detection device further includes:

[0075] A receiving module, used for receiving an assembly completion instruction;

[0076] An assembly information acquisition module, used to acquire assembly information of each finished product, wherein the finished product is assembled from at least one component in each tooling tray, and the assembly information at least includes assembly time and assembly pressure of the finished product;

[0077] The detection information acquisition module is used to obtain the detection information of each finished product by the preset detection equipment;

[0078] The traceability information generation module is used to generate traceability information and associate it with the preset codes of each component of the assembled product. The traceability information at least includes the assembly information and the detection information as well as the material category, production batch and actual model of each component of the assembled product.

[0079] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

[0080] An electronic device, comprising:

[0081] at least one processor;

[0082] Memory;

[0083] At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the above-mentioned production detection method.

[0084] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0085] A computer-readable storage medium includes: a computer program that can be loaded by a processor and execute the above-mentioned production detection method.

[0086] In summary, the present application includes at least one of the following beneficial technical effects:

[0087] By adopting the above technical solution, in the component conveying stage, the image to be inspected including all components in each tooling tray is obtained, and the model information of each component in each tray is obtained by target recognition, and then the standard model preset for each tooling tray is used to judge whether there are non-standard components in each tooling tray. When there are no non-standard components in all tooling trays, the robot is controlled to grab the components to the preset position for assembly, thereby improving the yield rate of the assembled finished product;

[0088] The generated traceability information is associated with the preset code on each component that makes up the finished product. This can effectively utilize the role of the damaged code and save time. By scanning the preset code of any component on the finished product and parsing it, the traceability information of the finished product in the production process can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 It is a schematic diagram of the process of the production detection method in the embodiment of the present application;

[0090] Figure 2 is a schematic diagram of an image angle of a component in any image to be inspected in the embodiments of the present application;

[0091] Figure 3 It is a structural schematic diagram of a production detection device in an embodiment of the present application;

[0092] Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0093] The following is combined with Figure 1 -Attached Figure 4 This application is described in further detail.

[0094] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.

[0095] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0096] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0097] The present application embodiment provides a production detection method, which is applied to a feeding system of at least two tooling trays and is executed by an electronic device, referring to Figure 1 The method comprises steps S101 to S104, wherein:

[0098] Step S101: Acquire an image to be inspected for each tooling tray, wherein each image to be inspected includes information of all components in the corresponding tooling tray.

[0099] In the embodiment of the present application, the image to be inspected is taken for all components placed in a tooling tray, and there is only one image to be inspected for each tooling tray. In order to facilitate the complete photographing of all components in each tooling tray, the image acquisition device should capture the image of the tooling tray from above the tooling tray at a top-down angle as the image to be inspected.

[0100] Step S102: Perform target recognition on each image to be inspected to obtain the actual model of each component in each tooling tray.

[0101] In actual production, the components on the tooling pallet are mostly placed manually by workers. Therefore, it may happen that the model placed in the same tooling pallet is different from the standard model preset for the pallet, which is the phenomenon of mixing materials.

[0102] In the embodiment of the present application, the signal information of the components is different, and the corresponding shape features or size features are also different. Therefore, by identifying the features of the components in the pre-inspection image, all the component targets in the image to be inspected are first confirmed, and then each component target is framed to obtain the framed area corresponding to each component target. Then, the features in each framed area are extracted for identification, and the actual model corresponding to each component can be obtained.

[0103] Step S103: based on the actual model of each component in each tooling pallet and the standard model corresponding to each tooling pallet, determine whether there are non-standard components in each tooling pallet. The actual model of the non-standard component does not match the standard model. Each tooling pallet is preset with a standard model for the component.

[0104] In the embodiment of the present application, a standard model is preset for each tooling pallet, that is, the actual models of all components in any tooling pallet should be standard models to meet the assembly process. At the same time, the non-standard components are determined by comparing the actual models of all components in the tooling pallet with the standard of the tooling pallet, and the components that do not conform to the standard model of the tooling pallet are determined as non-standard components. At the same time, when determining whether there are non-standard components in different tooling pallets, each tooling pallet can be tested in turn according to the preset order, and it can also be done simultaneously.

[0105] Step S104: If none of them exist, control the robot arm to grab the components in each tooling tray to a preset position.

[0106] In the embodiment of the present application, the manipulator grabs the components to be assembled and places them in a preset position for subsequent processing or direct assembly. The coordinates of the preset position can be pre-set. Please refer to the following embodiment for how the manipulator accurately identifies and controls the components in each tooling tray.

[0107] Furthermore, if there are non-standard components in any tooling tray, an alarm should be issued, that is, a mixed material alarm signal should be output. In order to facilitate users to find non-standard components in time and to facilitate timely replacement of non-standard components, the output alarm signal should include the standard model of the tooling tray where the non-standard components are located, and the number of non-standard components in the tooling tray where the non-standard components are located.

[0108] During the component conveying stage, images of all components in each tooling tray are obtained, and the model information of each component in each tray is obtained through target recognition. Then, the standard model preset for each tooling tray is used to determine whether there are non-standard components in each tooling tray. When there are no non-standard components in all tooling trays, the robot is controlled to grab the components to the preset position for assembly, thereby improving the yield rate of the assembled finished products.

[0109] Furthermore, before step S104, steps SA1 (not shown in the figure) to SA5 (not shown in the figure) are also included, wherein for each image to be inspected:

[0110] Step SA1: Acquire the size information and angle information of each component in the image to be inspected.

[0111] Specifically, different components have different size information. For example, for a cylindrical component, the size information may be the circumference of the cross section; for a rod-shaped component, the size information may be the length. Therefore, in the embodiment of the present application, there is no specific limitation on the user size information, and the user may make adaptive settings based on the specific shape of the component and the viewing angle of the image to be inspected.

[0112] Further, refer to Figure 2 For angle information, a reference line can be established based on the image to be inspected, and a reference point can be set up on the reference line to determine the geometric center of each component in the image to be inspected. For any component, the line connecting its corresponding geometric center in the image to be inspected and the reference point is the angle auxiliary line of the component, and the angle between the angle auxiliary line and the reference line is the angle information of the component. The method for determining the reference line and the reference point can be determined based on the layout position of the component on the tooling tray and the angle at which the image to be inspected is collected.

[0113] Step SA2: determining the actual size of each component in the image to be inspected based on the mapping relationship between the size information and the actual size of each component in the image to be inspected.

[0114] Specifically, for example, if the size information of an element is the cross-sectional circumference of the element, then the size information of the element is based on calculating the number of pixels occupied by the cross-sectional circumference of the element in the image to be inspected, and by presetting a mapping relationship between each pixel in the image to be inspected and the actual length, the actual size of the element can be determined based on the size information of the element and the mapping relationship.

[0115] Step SA3: determining the actual angle of each component in the image to be inspected based on the mapping relationship between the angle information of each component in the image to be inspected and the actual angle.

[0116] Specifically, each image acquisition device has a fixed shooting angle when acquiring the image to be inspected, that is, the framing angle can be a preset fixed value. Based on the mapping relationship between the angle information of the component and the preset angle information and the angle, combined with the framing angle, the actual angle of the component is obtained.

[0117] Step SA4: determining whether the actual size of each component in the image to be inspected is within the first preset interval, and determining whether the actual angle of each component in the image to be inspected is within the second preset interval;

[0118] Step SA5: If both are located at , control the robot arm to grab the components in the tooling tray to the preset position.

[0119] Specifically, only when the actual size corresponding to the component is within the first preset interval and the actual angle corresponding to the component is within the second preset interval, the component can meet the gripping standard of the manipulator. For the first preset interval and the second preset interval, the interval range needs to be set based on the specifications of the manipulator and the positional relationship between the manipulator and the tooling tray, and is not specifically limited in the embodiments of the present application.

[0120] Furthermore, the production detection method further includes step SP1 (not shown in the figure) and step SP2 (not shown in the figure), wherein:

[0121] Step SP1, control the preset device to scan the preset code printed on each component in each tooling tray to record the preset code on each component, and the preset code is associated with the production batch information of the corresponding component.

[0122] Specifically, before each component is placed in the corresponding tooling tray, a preset code should be printed or etched on the outer surface of the component, wherein the preset code can be a QR code or a barcode or other coding forms that can associate / store information.

[0123] Step SP2: parse the codes corresponding to the components to obtain the production batches corresponding to the components.

[0124] Specifically, the information associated with the preset code should be the production batch information of the component, so as to facilitate traceability when errors occur in the production process.

[0125] Furthermore, step SP1 and step SP2 may be performed before step S101 or after step S101. Different scanning devices are used to collect different forms of preset codes, which are not specifically limited here.

[0126] Furthermore, the production detection method further includes step SK1 (not shown in the figure) to step SK5 (not shown in the figure), wherein:

[0127] Step SK1, performing image segmentation on the image to be inspected to obtain target areas corresponding to each component in the image to be inspected;

[0128] Step SK2, extracting features from each target area to obtain feature information corresponding to each element, wherein the feature information includes at least color features and texture features;

[0129] Step SK3: determining the material category of each component based on the feature information corresponding to each component and the preset mapping relationship between the feature information and the material category.

[0130] Specifically, the material category corresponding to each component can be determined by color features and texture features. For components of the same size, they may be made of several different materials. Therefore, only the size of the component in step SA4 is used to determine whether it meets the gripping standard of the robot. However, it is still possible that the robot grips components of different materials, which will also lead to assembly failure.

[0131] Step SK4: determining whether the material categories of all components are preset standard categories;

[0132] Step SK5: If not, output category warning information.

[0133] Specifically, for any image to be inspected, after obtaining the material category of each component in the image to be inspected, the standard category preset by the tooling tray corresponding to the image to be inspected is judged. When the material category of at least one component does not match the standard category, the category format information is output; wherein the category information can be in text format, decoction piece format, image format, etc., but it must be different from the mixed material warning signal in step S104, so that the user can distinguish it.

[0134] Further, step S104 may include step S1041 (not shown in the figure) to step S1044 (not shown in the figure), wherein:

[0135] Step S1041 : obtaining the image coordinates of each component in each image to be inspected in the pre-inspection image.

[0136] Specifically, for each image to be inspected, step S1041 may include steps SN1 (not shown in the figure) and SN2 (not shown in the figure), wherein:

[0137] Step SN1, establishing an image coordinate system based on the preview image;

[0138] Step SN2: Determine the coordinates of each component in the preview image in the preview image coordinate system as the image coordinates corresponding to each component in the preview image.

[0139] Specifically, a rectangular coordinate system with the lower left corner as the origin, namely, the image coordinate system, is established for each image to be inspected; then the geometric center of the shape outline of each component in each component area in the image is used as the image coordinate of the component in the image to be inspected.

[0140] Step S1042, based on the image coordinates of each component in each preview image and the conversion relationship between the image coordinates and the actual coordinates of the component, determine the actual coordinates of each component in each preview image, the actual coordinates being the coordinates of the target component in the corresponding tooling tray.

[0141] Specifically, the angle and distance at which the image acquisition device acquires the image to be inspected should be fixed, so the conversion relationship between the image coordinates of the component in the image to be inspected and the actual coordinates of the component in the tooling tray should be fixed. After the image coordinates of the component are obtained through step S1041, the actual coordinates of the component are determined based on the image coordinates and the preset conversion relationship, so that the manipulator can grab the component based on the actual coordinates.

[0142] Step S1043, obtaining the assembly coordinates of the preset position;

[0143] Step S1044: Based on the actual coordinates of each component and the assembly coordinates, the robot is controlled to grab the components in each tooling tray to a preset position.

[0144] Specifically, the coordinates of the preset position are fixed. After obtaining the actual coordinates of the component to be grasped, the manipulator grasps the component and then delivers the component to the preset position. It should be noted that there can be only one manipulator, and the components are grasped in a preset order. There can also be multiple manipulators, that is, multiple components can be grasped simultaneously.

[0145] Furthermore, after the manipulator grabs the component to the preset position, the preset assembly equipment assembles the component into a finished product, wherein the finished product is assembled from at least one component in each tooling tray. After the finished product is assembled, it also needs to be inspected by the preset inspection equipment. After the inspection is completed, the electronic equipment can obtain the inspection information of each finished product by the inspection equipment.

[0146] In order to facilitate the traceability of the components of the assembled finished product, the production inspection method further includes steps SF1 (not shown in the figure) to SF4 (not shown in the figure), wherein:

[0147] Step SF1, receiving an assembly completion instruction;

[0148] Step SF2: Acquire assembly information of each finished product. The finished product is assembled from at least one component in each tooling tray. The assembly information at least includes assembly time and assembly pressure of the finished product.

[0149] Specifically, the assembly completion instruction can be input by the user, or automatically output by the testing equipment when it starts testing the finished product, and then received by the electronic device. The assembly information represents the time and pressure environment of the finished product assembly, which facilitates the subsequent traceability of the assembly nodes of the finished product.

[0150] Step SF3: obtaining the detection information of each finished product by the preset detection equipment;

[0151] Step SF4: Generate traceability information and associate it with the preset codes of the components of the finished assembly. The traceability information includes at least assembly information and inspection information as well as the material category, production batch and actual model of the components of the finished assembly.

[0152] Specifically, the detection information can be input by the user, or it can be output by the detection equipment when the detection of each finished product component is completed, and then received by the electronic device. The traceability information of each finished product should include the material category, production batch and actual model of each component that constitutes the finished product; it also includes the assembly information and detection information of the finished product itself. At the same time, in order to reduce costs and time, the traceability information is associated with the preset code of the outer wall of each component of the assembled finished product, that is, after scanning and recording the preset code, the traceability information of the finished product can be parsed.

[0153] The above embodiment introduces a production detection method from the perspective of method flow, and the following embodiment introduces a production detection device from the perspective of a virtual module or a virtual unit. Please refer to the following embodiment for details.

[0154] The present application embodiment provides a production detection device, such as Figure 3As shown, the device 300 may specifically include a to-be-detected image acquisition module 301, a target recognition module 302, a first judgment module 303 and a control module 304, wherein:

[0155] The inspection image acquisition module 301 is used to acquire an inspection image collected for each tooling tray, and each inspection image includes all component information in the corresponding tooling tray;

[0156] The target recognition module 302 is used to perform target recognition on each image to be inspected and obtain the actual model of each component in each tooling tray;

[0157] The first judgment module 303 is used to judge whether there are non-standard components in each tooling tray based on the actual model of each component in each tooling tray and the standard model corresponding to each tooling tray. The actual model of the non-standard component does not match the standard model. Each tooling tray is preset with a standard model for the component;

[0158] The control module 304 is used to control the robot to grab the components in each tooling tray to a preset position.

[0159] In a possible implementation, the production detection device further includes:

[0160] A first acquisition module is used to acquire the size information and angle information of each component in the image to be inspected;

[0161] An actual size determination module, used to determine the actual size of each component in the image to be inspected based on the mapping relationship between the size information of each component in the image to be inspected and the actual size;

[0162] An actual angle determination module, used to determine the actual angle of each component in the image to be inspected based on the mapping relationship between the angle information of each component in the image to be inspected and the actual angle;

[0163] The second judgment module is used to judge whether the actual size of each component in the image to be inspected is within the first preset interval, and to judge whether the actual angle of each component in the image to be inspected is within the second preset interval.

[0164] In a possible implementation, the production detection device further includes:

[0165] Control the preset device to scan the preset code printed on each component in each tooling tray to record the preset code on each component, and the preset code is associated with the production batch information of the corresponding component;

[0166] The codes corresponding to each component are parsed to obtain the production batches corresponding to each component.

[0167] In a possible implementation, the production device further includes:

[0168] An image segmentation module is used to segment the image to be inspected to obtain target areas corresponding to each component in the image to be inspected;

[0169] A feature extraction module is used to extract features from each target area to obtain feature information corresponding to each element, wherein the feature information includes at least color features and texture features;

[0170] A material category determination module, used to determine the material category of each component based on the feature information corresponding to each component and a preset mapping relationship between the feature information and the material category;

[0171] A third judgment module is used to judge whether the material category of all components is a preset standard category;

[0172] The warning information output module is used to output category warning information.

[0173] In a possible implementation, when the control module 304 controls the manipulator to grab the preset position of the component in each tooling tray, it is specifically used to:

[0174] Obtaining the image coordinates of each component in each image to be inspected in the pre-inspection image;

[0175] Determine the actual coordinates of each component in each pre-inspection image based on the image coordinates of each component in each pre-inspection image and the conversion relationship between the image coordinates and the actual coordinates of the component, where the actual coordinates are the coordinates of the target component in the corresponding tooling tray;

[0176] Get the assembly coordinates of the preset position;

[0177] Based on the actual coordinates of each component and the assembly coordinates, the robot is controlled to grab the components in each tooling tray to the preset position.

[0178] In a possible implementation, when the control module 304 obtains the image coordinates of each component in each preview image, for any preview image, it is specifically used to:

[0179] Establishing an image coordinate system based on the preview image;

[0180] The coordinates of each component in the preview image in the preview image coordinate system are determined as image coordinates corresponding to each component in the preview image.

[0181] In a possible implementation, the production detection device further includes:

[0182] A receiving module, used for receiving an assembly completion instruction;

[0183] An assembly information acquisition module, used to acquire assembly information of each finished product, where the finished product is assembled from at least one component in each tooling tray, and the assembly information at least includes assembly time and assembly pressure of the finished product;

[0184] The detection information acquisition module is used to obtain the detection information of each finished product by the preset detection equipment;

[0185] The traceability information generation module is used to generate traceability information and associate it with the preset codes of each component of the assembled product. The traceability information at least includes assembly information and detection information as well as the material category, production batch and actual model of each component of the assembled product.

[0186] An electronic device is provided in an embodiment of the present application, such as Figure 4 As shown, Figure 4 The electronic device 400 shown includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as through a bus 402. Optionally, the electronic device 400 may also include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one, and the structure of the electronic device 400 does not constitute a limitation on the embodiments of the present application.

[0187] Processor 401 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0188] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0189] The memory 403 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0190] The memory 403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement the contents shown in the above method embodiment.

[0191] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0192] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0193] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0194] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A production detection method, characterized in that: A feeding system applied to at least two tooling pallets, comprising: obtaining an image to be inspected collected for each tooling pallet, each of the image to be inspected including all component information in the corresponding tooling pallet; performing target recognition on each of the image to be inspected to obtain the actual model of each component in each tooling pallet; judging whether there is a non-standard component in each tooling pallet based on the actual model of each component in each tooling pallet and the standard model corresponding to each tooling pallet, wherein the actual model of the non-standard component does not match the standard model, and each tooling pallet is preset with a standard model for the component; obtaining size information and angle information of each component in the image to be inspected; determining the actual size of each component in the image to be inspected based on the mapping relationship between the size information of each component in the image to be inspected and the actual size; determining the actual angle of each component in the image to be inspected based on the mapping relationship between the angle information of each component in the image to be inspected and the actual angle; judging whether the actual size of each component in the image to be inspected is within a first preset interval, and judging whether the actual angle of each component in the image to be inspected is within a second preset interval; if both do not exist and both are within, controlling a manipulator to grab the component in the tooling pallet to a preset position.

2. A production detection method according to claim 1, characterized in that: Also includes: Controlling the preset device to scan the preset code printed on each component in each tooling tray to record the preset code on each component, wherein the preset code is associated with the production batch information of the corresponding component; The codes corresponding to the components are parsed to obtain the production batches corresponding to the components.

3. A production detection method according to claim 1, characterized in that: For any image to be inspected, it also includes: performing image segmentation on the image to be inspected to obtain target areas corresponding to each component in the image to be inspected; performing feature extraction on each target area to obtain feature information corresponding to each component, wherein the feature information at least includes color features and texture features; determining the material category of each component based on the feature information corresponding to each component and a mapping relationship between preset feature information and material categories; judging whether the material categories of all the components are preset standard categories; if not, outputting category warning information.

4. A production detection method according to claim 1, characterized in that: The method of controlling the manipulator to grab the preset position of the component in each tooling tray includes: obtaining the image coordinates of each component in each of the images to be inspected in the pre-inspection image; determining the actual coordinates of each component in each of the pre-inspection images based on the image coordinates of each component in the pre-inspection image and the conversion relationship between the image coordinates and the actual coordinates of the component, wherein the actual coordinates are the coordinates of the target component in the corresponding tooling tray; obtaining the assembly coordinates of the preset position; and controlling the manipulator to grab the component in each tooling tray to the preset position based on the actual coordinates of each component and the assembly coordinates.

5. A production detection method according to claim 4, characterized in that: The obtaining of the image coordinates of each component in each preview image, for any preview image, includes: establishing an image coordinate system based on the preview image; and determining the coordinates of each component in the preview image in the preview image coordinate system as the image coordinates corresponding to each component in the preview image.

6. The production detection method according to any one of claims 1 to 4, characterized in that: Also includes: Receive an assembly completion instruction; obtain assembly information of each finished product, wherein the finished product is assembled from at least one component in each tooling tray, and the assembly information at least includes the assembly time and assembly pressure of the finished product; obtain detection information of each finished product by a preset detection device; generate traceability information and associate it with a preset code of each component of the assembled finished product, wherein the traceability information at least includes the assembly information and the detection information as well as the material category, the production batch and the actual model of each component of the assembled finished product.

7. A production detection device, characterized in that: include: An image acquisition module for inspection, used to acquire an image for inspection collected for each tooling tray, each of which includes information of all components in the corresponding tooling tray; A target recognition module is used to perform target recognition on each of the images to be inspected to obtain the actual model of each component in each tooling tray; A first judgment module is used to judge whether there is a non-standard component in each tooling tray based on the actual model of each component in each tooling tray and the standard model corresponding to each tooling tray, and the actual model of the non-standard component does not match the standard model, and each tooling tray is preset with a standard model for the component; The first judgment module is further used to obtain the size information and angle information of each component in the image to be inspected; determine the actual size of each component in the image to be inspected based on the mapping relationship between the size information and the actual size of each component in the image to be inspected; Determining the actual angle of each component in the image to be inspected based on the mapping relationship between the angle information of each component in the image to be inspected and the actual angle; Determine whether the actual size of each component in the image to be inspected is within a first preset interval, and determine whether the actual angle of each component in the image to be inspected is within a second preset interval; a control module, used to control the robot to grab the components in each tooling tray to a preset position when the judgment results of the first judgment module are that both do not exist and are located.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; a memory; and at least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, and the at least one application is configured to: execute the production detection method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: include: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 6.

Images