Industrial Vision Automatic Alignment Method and Related Equipment

Abstract

This application relates to the field of industrial vision inspection technology, and provides an industrial vision automatic alignment method and related equipment. The method includes: collecting visible light images, point cloud data information, and hyperspectral images of the target to be aligned; determining the part name of the target to be aligned based on the visible light image and the hyperspectral image; obtaining the virtual alignment scene of the target to be aligned in the database based on the part name; wherein the virtual alignment scene includes standard virtual parts of the target to be aligned; constructing virtual parts of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; controlling the virtual parts to move to coincide with the standard virtual parts in the virtual alignment scene, and recording the movement trajectory of the virtual parts during the process of controlling the movement of the virtual parts; performing alignment processing on the target to be aligned based on the movement trajectory. This method helps to achieve high-precision alignment of small and complex parts.

Description

Technical Field

[0001] This application relates to the field of industrial vision inspection technology, and particularly to an industrial vision automatic alignment method and related equipment. Background Art

[0002] With the rapid development of industrial automation, the requirements for the accuracy, speed, and stability of alignment operations in the production and manufacturing process are continuously increasing. Traditional alignment technologies usually rely on manual operations or simple vision positioning systems, but their alignment accuracy shows obvious deficiencies in complex alignment environments. Especially when performing high-precision alignment on small and complex components, traditional alignment methods are difficult to meet production requirements. Summary of the Invention

[0003] This application provides an industrial vision automatic alignment method and related equipment to solve the problems raised in the above background art.

[0004] In a first aspect, this application provides an industrial vision automatic alignment method, including:

[0005] Collecting multi-modal image information of a target to be aligned through a multi-modal image acquisition device at a preset position; wherein, the multi-modal image information includes visible light images, point cloud data information, and hyperspectral images;

[0006] Determining the part name of the target to be aligned based on the visible light image and the hyperspectral image;

[0007] Obtaining a virtual alignment scene of the target to be aligned from a database based on the part name; wherein, the virtual alignment scene includes a standard virtual part of the target to be aligned;

[0008] Constructing a virtual part of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information;

[0009] Controlling the virtual part to move to coincide with the standard virtual part in the virtual alignment scene, and recording the movement trajectory of the virtual part during the process of controlling the movement of the virtual part;

[0010] Controlling a manipulator to perform alignment processing on the target to be aligned based on the movement trajectory.

[0011] In a possible implementation, before obtaining the virtual alignment scene of the target to be aligned from the database based on the part name, the method further includes:

[0012] Obtaining the standard hyperspectral image of the target to be aligned from the database based on the part name;

[0013] Judge whether there is a defect in the target to be aligned based on the hyperspectral image and the standard hyperspectral image;

[0014] If not, execute the steps after determining the component name of the target to be aligned based on the visible light image and the hyperspectral image.

[0015] In a possible implementation manner, the judging whether there is a defect in the target to be aligned based on the hyperspectral image and the standard hyperspectral image includes:

[0016] Obtain the first virtual model of the component corresponding to the standard hyperspectral image in the database, and construct the second virtual model of the target to be aligned based on the visible light image and the point cloud data information;

[0017] Adjust the pose of the second virtual model to make the pose of the second virtual model consistent with the pose of the first virtual model, and record the pose adjustment parameter information during the process of adjusting the pose of the second virtual model;

[0018] Adjust the pose of the hyperspectral image based on the pose adjustment parameter information;

[0019] Construct a first space rectangular coordinate system with the center of gravity point of the standard hyperspectral image as the origin, and construct a second space rectangular coordinate system with the center of gravity point of the pose-adjusted hyperspectral image as the origin;

[0020] Perform segmentation processing on the standard hyperspectral image in the first space rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and perform segmentation processing on the pose-adjusted hyperspectral image in the second space rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks;

[0021] Judge whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks;

[0022] If not, determine that there is a defect in the target to be aligned;

[0023] If consistent, for each of the second spectral image blocks, judge whether the second spectral image block has a defect based on each of the first spectral image blocks;

[0024] If none of the second spectral image blocks has a defect, determine that the target to be aligned has no defect;

[0025] If any one of the second spectral image blocks has a defect, determine that the target to be aligned has a defect.

[0026] In a possible implementation, determining whether the second spectral image block has a defect based on each of the first spectral image blocks includes:

[0027] Determining whether there is a target first spectral image block corresponding to the second spectral image block in each of the first spectral image blocks; wherein, the first coordinate point corresponding to the center of gravity of the target first spectral image block in the first space rectangular coordinate system is the same as the second coordinate point corresponding to the second spectral image block in the second space rectangular coordinate system;

[0028] If there is, obtaining the similarity between the target first spectral image block and the second spectral image block, and comparing the similarity with a preset similarity;

[0029] If the similarity is greater than the preset similarity, determining that the second spectral image block has no defect;

[0030] If the similarity is not greater than the preset similarity, determining that the second spectral image block has a defect.

[0031] In a possible implementation, constructing the virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information includes:

[0032] Constructing the virtual geometric structure of the target to be aligned in the virtual alignment scene based on the point cloud data information;

[0033] Rendering the virtual geometric structure based on the visible light image to obtain the virtual component.

[0034] In a second aspect, the present application provides an industrial vision automatic alignment device, including:

[0035] An acquisition module, configured to acquire multi-modal image information of a target to be aligned through a multi-modal image acquisition device at a preset position; wherein, the multi-modal image information includes a visible light image, point cloud data information, and a hyperspectral image;

[0036] A determination module, configured to determine the component name of the target to be aligned based on the visible light image and the hyperspectral image;

[0037] An acquisition module, configured to acquire the virtual alignment scene of the target to be aligned in a database based on the component name; wherein, the virtual alignment scene includes a standard virtual component of the target to be aligned;

[0038] A construction module, configured to construct the virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information;

[0039] A control module, configured to control the virtual component to move to coincide with the standard virtual component in the virtual alignment scenario, and record the movement trajectory of the virtual component during the process of controlling the movement of the virtual component;

[0040] An alignment processing module, configured to control a manipulator to perform alignment processing on the target to be aligned based on the movement trajectory.

[0041] In a third aspect, the present application provides a terminal device, which includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the computer program is executed by the processor, the industrial vision automatic alignment method described in any one of the above is implemented.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the industrial vision automatic alignment method described in any one of the above is implemented.

[0043] The present application provides an industrial vision automatic alignment method and related devices. The method includes: collecting multi-modal image information of a target to be aligned through a multi-modal image acquisition device at a preset position; where the multi-modal image information includes a visible light image, point cloud data information, and a hyperspectral image; determining the component name of the target to be aligned based on the visible light image and the hyperspectral image; obtaining a virtual alignment scenario of the target to be aligned from a database based on the component name; where the virtual alignment scenario includes a standard virtual component of the target to be aligned; constructing a virtual component of the target to be aligned in the virtual alignment scenario based on the visible light image and the point cloud data information; controlling the virtual component to move to coincide with the standard virtual component in the virtual alignment scenario, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; controlling a manipulator to perform alignment processing on the target to be aligned based on the movement trajectory. On the one hand, by determining the component name of the target to be aligned based on the visible light image and the hyperspectral image, components with similar appearances but different chemical compositions can be accurately distinguished, which helps to prevent confusion of components with similar appearances but different chemical compositions, thereby improving the accuracy and precision of the alignment result. On the other hand, by controlling the virtual component to move to coincide with the standard virtual component in the virtual alignment scenario and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component, an accurate simulation of the alignment process of the target to be aligned is realized, which helps to improve the precision during the actual alignment operation process and achieve high-precision alignment of small and complex components. Description of the Drawings

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0045] Figure 1 It is a schematic flowchart of the industrial vision automatic alignment method provided by the embodiments of this application;

[0046] Figure 2 It is a schematic block diagram of the structure of the industrial vision automatic alignment device provided by the embodiments of this application;

[0047] Figure 3 It is a schematic block diagram of the structure of the terminal device provided by the embodiments of this application. Specific embodiments

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0049] The flowchart shown in the accompanying drawings is only an example illustration, and does not necessarily include all content and operations / steps, nor does it necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged. Therefore, the actual execution order may change according to the actual situation.

[0050] It should also be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification of this application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0051] It should be further understood that the term " / and" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0052] The following will describe in detail some embodiments of this application in conjunction with the accompanying drawings. Without conflict, the features in the following embodiments and the embodiments can be combined with each other.

[0053] Please refer to Figure 1 , Figure 1The flowchart of the industrial vision automatic alignment method provided by the embodiment of the present application is shown as Figure 1 shown. The industrial vision automatic alignment method provided by the embodiment of the present application includes steps S1 to S6.

[0054] Step S1: Collect multi-modal image information of the target to be aligned through a multi-modal image acquisition device at a preset position; wherein, the multi-modal image information includes visible light images, point cloud data information, and hyperspectral images.

[0055] Among them, the multi-modal image acquisition device includes a plurality of multi-modal image acquisition units, each of the multi-modal image acquisition units is arranged at a different position, and each of the multi-modal image acquisition units respectively collects multi-modal images of different parts of the target to be aligned to obtain the multi-modal image information. The multi-modal image information includes the global multi-modal image of the target to be aligned. Each of the multi-modal image acquisition units includes a camera, a lidar, and a hyperspectral camera. Among them, the camera is used to collect visible light images, the lidar is used to collect point cloud data information, and the hyperspectral camera is used to collect hyperspectral images.

[0056] Step S2: Determine the part name of the target to be aligned based on the visible light image and the hyperspectral image.

[0057] Specifically, input the visible light image and the hyperspectral image into a preset part recognition model to obtain the part name of the target to be aligned. The part recognition model is a neural network model that has been pre-trained.

[0058] It can be understood that the hyperspectral image helps to identify the chemical composition information of the target to be aligned. By determining the part name of the target to be aligned based on the visible light image and the hyperspectral image in step S2, it is possible to accurately distinguish parts with similar appearances but different chemical compositions, which helps to prevent confusion of parts with similar appearances but different chemical compositions, thereby improving the accuracy and precision of the alignment result.

[0059] Step S3: Obtain the virtual alignment scene of the target to be aligned in the database based on the part name; wherein, the virtual alignment scene includes the standard virtual part of the target to be aligned.

[0060] Among them, the corresponding relationship between the part name and the virtual alignment scene is stored in the database. The method for obtaining the virtual alignment scene of the target to be aligned is to accurately align the part identical to the target to be aligned in the alignment scene corresponding to the multi-modal image acquisition device, and then obtain the standard multi-modal image information of the alignment scene through the multi-modal image acquisition device, and construct the virtual alignment scene based on the standard visible light image and the standard point cloud data information in the standard multi-modal image information.

[0061] Step S4: Construct the virtual part of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information.

[0062] Specifically, step S4 includes the following steps:

[0063] Construct the virtual geometric structure of the target to be aligned in the virtual alignment scene based on the point cloud data information; specifically, for each point in the point cloud data information, map each point to the virtual alignment scene based on the position of the point in the alignment scene to obtain the virtual geometric structure of the target to be aligned;

[0064] Render the virtual geometric structure based on the visible light image to obtain the virtual part.

[0065] Step S5: Control the virtual part to move to coincide with the standard virtual part in the virtual alignment scene, and record the movement trajectory of the virtual part during the process of controlling the movement of the virtual part.

[0066] Step S6: Control the manipulator to perform alignment processing on the target to be aligned based on the movement trajectory.

[0067] For the method provided in this embodiment, on the one hand, by determining the part name of the target to be aligned based on the visible light image and the hyperspectral image, it is possible to accurately distinguish parts with similar appearances but different chemical compositions, which helps to prevent confusion of parts with similar appearances but different chemical compositions, thereby improving the accuracy and precision of the alignment result. On the other hand, by controlling the virtual part to move to coincide with the standard virtual part in the virtual alignment scene and recording the movement trajectory of the virtual part during the process of controlling the movement of the virtual part, an accurate simulation of the alignment process of the target to be aligned is realized, which helps to improve the precision in the actual alignment operation process and achieve high-precision alignment of small and complex parts.

[0068] In some embodiments, before obtaining the virtual alignment scene of the target to be aligned in the database based on the part name, the method further includes the following steps:

[0069] Obtain the standard hyperspectral image of the to-be-aligned target in the database based on the component name;

[0070] Judge whether there are defects in the to-be-aligned target based on the hyperspectral image and the standard hyperspectral image;

[0071] If not, execute the steps after determining the component name of the to-be-aligned target based on the visible light image and the hyperspectral image.

[0072] In this embodiment, judging whether there are defects in the to-be-aligned target based on the hyperspectral image and the standard hyperspectral image includes the following steps:

[0073] Obtain the first virtual model of the component corresponding to the standard hyperspectral image in the database, and construct the second virtual model of the to-be-aligned target based on the visible light image and the point cloud data information; it can be understood that the component and the to-be-aligned target belong to the same type of component, the component has no defects, and the first virtual model is a virtual model constructed based on the posture of the component and consistent with the component when obtaining the standard hyperspectral image of the component;

[0074] Adjust the posture of the second virtual model to make the posture of the second virtual model consistent with the posture of the first virtual model, and record the posture adjustment parameter information during the process of adjusting the posture of the second virtual model; wherein, the posture adjustment parameter information includes the rotation direction and the rotation angle corresponding to the rotation direction, the rotation direction includes at least one, and when adjusting the pose of the second virtual model, rotate the second virtual model with the central axis of the second virtual model as the rotation axis;

[0075] Adjust the posture of the hyperspectral image based on the posture adjustment parameter information;

[0076] Construct a first spatial rectangular coordinate system with the center of gravity point of the standard hyperspectral image as the origin, and construct a second spatial rectangular coordinate system with the center of gravity point of the posture-adjusted hyperspectral image as the origin;

[0077] Perform segmentation processing on the standard hyperspectral image in the first spatial rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and perform segmentation processing on the posture-adjusted hyperspectral image in the second spatial rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks;

[0078] Judge whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks;

[0079] If they are inconsistent, determine that there are defects in the to-be-aligned target;

[0080] If they are consistent, for each of the second spectral image blocks, it is determined whether there are defects in the second spectral image block based on each of the first spectral image blocks;

[0081] If there are no defects in each of the second spectral image blocks, it is determined that there are no defects in the target to be aligned;

[0082] If there is any defect in any of the second spectral image blocks, it is determined that there are defects in the target to be aligned.

[0083] In this embodiment, the determining whether there are defects in the second spectral image block based on each of the first spectral image blocks includes the following steps:

[0084] It is determined whether there is a target first spectral image block corresponding to the second spectral image block in each of the first spectral image blocks; wherein, the first coordinate point corresponding to the center of gravity point of the target first spectral image block in the first space rectangular coordinate system is consistent with the second coordinate point corresponding to the second spectral image block in the second space rectangular coordinate system;

[0085] If there is, the similarity between the target first spectral image block and the second spectral image block is obtained, and the similarity is compared with a preset similarity; specifically, the target first spectral image block is input into a preset spectral image block feature extraction model to obtain a first feature vector, and the second spectral image block is input into the spectral image block feature extraction model to obtain a second feature vector, and the cosine value of the first feature vector and the second feature vector is determined as the similarity;

[0086] If the similarity is greater than the preset similarity, it is determined that there are no defects in the second spectral image block;

[0087] If the similarity is not greater than the preset similarity, it is determined that there are defects in the second spectral image block.

[0088] It can be understood that the method for determining whether there are defects in the target to be aligned based on the hyperspectral image and the standard hyperspectral image realizes a refined defect detection method, which helps to improve the accuracy of the defect detection result.

[0089] The method provided in this embodiment can pre-detect possible defects in the target to be aligned by comparing the hyperspectral image of the target to be aligned with the standard hyperspectral image before aligning the target to be aligned, which helps to effectively avoid defective components from entering the subsequent alignment process, reduces resource waste and improves the reliability of the overall process.

[0090] Please refer to Figure 2 ,Figure 2 This is a schematic block diagram of the industrial vision automatic alignment device 100 provided by an embodiment of the present application. As Figure 2 shown, the industrial vision automatic alignment device 100 provided by an embodiment of the present application includes:

[0091] An acquisition module 110, configured to acquire multi-modal image information of a target to be aligned through a multi-modal image acquisition device at a preset position; wherein, the multi-modal image information includes a visible light image, point cloud data information, and a hyperspectral image.

[0092] A determination module 120, configured to determine the component name of the target to be aligned based on the visible light image and the hyperspectral image.

[0093] An acquisition module 130, configured to acquire a virtual alignment scene of the target to be aligned in a database based on the component name; wherein, the virtual alignment scene includes a standard virtual component of the target to be aligned.

[0094] A construction module 140, configured to construct a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information.

[0095] A control module 150, configured to control the virtual component to move to coincide with the standard virtual component in the virtual alignment scene, and record the movement trajectory of the virtual component during the process of controlling the movement of the virtual component.

[0096] An alignment processing module 160, configured to control a manipulator to perform alignment processing on the target to be aligned based on the movement trajectory.

[0097] It should be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described device and each module can refer to the corresponding processes in the foregoing embodiments of the industrial vision automatic alignment method, and will not be elaborated herein.

[0098] The terminal device 200 provided by the above embodiment can be implemented in the form of a computer program, and the computer program can run on the terminal device 200 as Figure 3 shown.

[0099] Please refer to Figure 3 , Figure 3 This is a schematic block diagram of the terminal device 200 provided by an embodiment of the present application. The terminal device 200 includes a processor 201 and a memory 202. The processor 201 and the memory 202 are connected through a device bus 203. Among them, the memory 202 can include a non-volatile storage medium and an internal memory.

[0100] The non-volatile storage medium can store a computer program. The computer program includes program instructions that, when executed by the processor 201, can cause the processor 201 to execute any of the above industrial vision automatic alignment methods.

[0101] The processor 201 is used to provide computing and control capabilities to support the operation of the entire terminal device 200.

[0102] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 201, it can cause the processor 201 to execute any of the above industrial vision automatic alignment methods.

[0103] Those skilled in the art can understand that Figure 3 The structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the terminal device 200 involved in the solution of this application. The specific terminal device 200 may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0104] It should be understood that the processor 201 can be a central processing unit (CPU), and the processor 201 can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0105] Among them, in some embodiments, the processor 201 is used to run the computer program stored in the memory to implement the following steps:

[0106] Collect multi-modal image information of the target to be aligned through a multi-modal image acquisition device at a preset position; wherein, the multi-modal image information includes visible light images, point cloud data information, and hyperspectral images;

[0107] Determine the component name of the target to be aligned based on the visible light image and the hyperspectral image;

[0108] Obtain the virtual alignment scene of the target to be aligned from the database based on the component name; wherein, the virtual alignment scene includes the standard virtual components of the target to be aligned;

[0109] Construct a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information;

[0110] In the virtual alignment scene, control the virtual component to move to coincide with the standard virtual component, and record the movement trajectory of the virtual component during the process of controlling the movement of the virtual component;

[0111] Based on the movement trajectory, control the manipulator to perform alignment processing on the target to be aligned.

[0112] It should be noted that those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described terminal device 200 can refer to the corresponding process of the foregoing industrial vision automatic alignment method, which will not be elaborated here.

[0113] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the one or more processors are caused to implement the industrial vision automatic alignment method provided by the embodiment of the present application.

[0114] Among them, the computer-readable storage medium may be an internal storage unit of the foregoing embodiment of the terminal device 200, such as the hard disk or memory of the terminal device 200. The computer-readable storage medium may also be an external storage device of the terminal device 200, such as a plug-in hard disk equipped with the terminal device 200, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc.

[0115] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present application, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An industrial vision automatic alignment method, characterized in that: include: Collecting multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; Determining a component name of the target to be aligned based on the visible light image and the hyperspectral image; Acquire a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; Constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; Controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; Controlling the manipulator to perform alignment processing on the target to be aligned based on the moving trajectory; Wherein, before acquiring the virtual alignment scene of the target to be aligned in the database based on the component name, the method further includes: Acquire a standard hyperspectral image of the target to be aligned in a database based on the component name; Determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image; If not, executing the steps after determining the component name of the target to be aligned based on the visible light image and the hyperspectral image; The determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image includes: Acquire a first virtual model of a component corresponding to the standard hyperspectral image in a database, and construct a second virtual model of the target to be aligned based on the visible light image and the point cloud data information; Performing posture adjustment on the second virtual model so that the posture of the second virtual model is consistent with the posture of the first virtual model, and recording posture adjustment parameter information in the process of performing posture adjustment on the second virtual model; Performing attitude adjustment on the hyperspectral image based on the attitude adjustment parameter information; A first spatial rectangular coordinate system is constructed with the center of gravity of the standard hyperspectral image as the origin, and a second spatial rectangular coordinate system is constructed with the center of gravity of the hyperspectral image after the posture adjustment as the origin; The standard hyperspectral image is segmented in the first spatial rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and the posture-adjusted hyperspectral image is segmented in the second spatial rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks; determining whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks; If they are inconsistent, it is determined that the target to be aligned has defects; If they are consistent, for each of the second spectral image blocks, judging whether the second spectral image block has a defect based on each of the first spectral image blocks; If none of the second spectral image blocks has defects, it is determined that the object to be aligned has no defects; If any of the second spectral image blocks has a defect, it is determined that the object to be aligned has a defect.

2. The industrial vision automatic alignment method according to claim 1, characterized in that: The determining whether the second spectral image block has a defect based on each of the first spectral image blocks includes: Determine whether there is a target first spectral image block corresponding to the second spectral image block in each of the first spectral image blocks; wherein a first coordinate point corresponding to the center of gravity of the target first spectral image block in the first spatial rectangular coordinate system is consistent with a second coordinate point corresponding to the second spectral image block in the second spatial rectangular coordinate system; If so, obtaining the similarity between the target first spectral image block and the second spectral image block, and comparing the similarity with a preset similarity; If the similarity is greater than the preset similarity, it is determined that the second spectral image block has no defects; If the similarity is not greater than the preset similarity, it is determined that the second spectral image block has defects.

3. The industrial vision automatic alignment method according to claim 1, characterized in that: The step of constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information includes: Constructing a virtual geometric structure of the target to be aligned in the virtual alignment scene based on the point cloud data information; The virtual geometric structure is rendered based on the visible light image to obtain the virtual component.

4. An industrial vision automatic alignment device, characterized in that: include: An acquisition module, used to acquire multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; A determination module, used to determine the component name of the target to be aligned based on the visible light image and the hyperspectral image; An acquisition module, used for acquiring a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; A construction module, used for constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; A control module, used for controlling the movement of the virtual component to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; An alignment processing module, used for controlling the manipulator to perform alignment processing on the target to be aligned based on the movement trajectory; Wherein, before acquiring the virtual alignment scene of the target to be aligned in the database based on the component name, the device is further used for: Acquire a standard hyperspectral image of the target to be aligned in a database based on the component name; Determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image; If not, executing the steps after determining the component name of the target to be aligned based on the visible light image and the hyperspectral image; The determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image includes: Acquire a first virtual model of a component corresponding to the standard hyperspectral image in a database, and construct a second virtual model of the target to be aligned based on the visible light image and the point cloud data information; Performing posture adjustment on the second virtual model so that the posture of the second virtual model is consistent with the posture of the first virtual model, and recording posture adjustment parameter information in the process of performing posture adjustment on the second virtual model; Performing attitude adjustment on the hyperspectral image based on the attitude adjustment parameter information; A first spatial rectangular coordinate system is constructed with the center of gravity of the standard hyperspectral image as the origin, and a second spatial rectangular coordinate system is constructed with the center of gravity of the hyperspectral image after the posture adjustment as the origin; The standard hyperspectral image is segmented in the first spatial rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and the posture-adjusted hyperspectral image is segmented in the second spatial rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks; determining whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks; If they are inconsistent, it is determined that the target to be aligned has defects; If they are consistent, for each of the second spectral image blocks, judging whether the second spectral image block has a defect based on each of the first spectral image blocks; If none of the second spectral image blocks has defects, it is determined that the object to be aligned has no defects; If any of the second spectral image blocks has a defect, it is determined that the object to be aligned has a defect.

5. A terminal device, characterized in that: The terminal device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the industrial vision automatic alignment method as described in any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the industrial vision automatic alignment method according to any one of claims 1 to 3 is implemented.

Images