Industrial vision automatic alignment method and related equipment

Abstract

The invention relates to the technical field of industrial vision detection, and provides an industrial vision automatic alignment method and related equipment, and the method comprises the steps: collecting a visible light image, point cloud data information and a hyperspectral image of a to-be-aligned target; determining the component name of the to-be-aligned target based on the visible light image and the hyperspectral image; acquiring a virtual alignment scene of the to-be-aligned target in a database based on the component name; wherein the virtual alignment scene comprises a standard virtual part of the target to be aligned; constructing a virtual part of the to-be-aligned target in the virtual alignment scene based on the visible light image and the point cloud data information; controlling the virtual part to move to be overlapped with the standard virtual part in the virtual alignment scene, and recording the moving track of the virtual part in the process of controlling the virtual part to move; and performing alignment processing on the to-be-aligned target based on the moving track. According to the method, high-precision alignment of small and complex parts can be realized.

Description

Technical Field

[0001] The present application relates to the field of industrial vision detection technology, and in particular to an industrial vision automatic alignment method and related equipment. Background Art

[0002] With the rapid development of industrial automation, the requirements for the accuracy, speed and stability of alignment operations in the manufacturing process are constantly increasing. Traditional alignment technology usually relies on manual operation or simple visual positioning systems, but its alignment accuracy is obviously insufficient in complex alignment environments. Especially when high-precision alignment is performed on small and complex parts, traditional alignment methods are difficult to meet production needs. Summary of the invention

[0003] The present application provides an industrial vision automatic alignment method and related equipment to solve the problems raised by the above background technology.

[0004] In a first aspect, the present application provides an industrial vision automatic alignment method, comprising: Collecting multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; Determining a component name of the target to be aligned based on the visible light image and the hyperspectral image; Acquire a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; Constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; Controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; Based on the movement trajectory, the robot is controlled to perform alignment processing on the target to be aligned.

[0005] In a possible implementation manner, before acquiring the virtual alignment scene of the target to be aligned in the database based on the component name, the method further includes: Acquire a standard hyperspectral image of the target to be aligned in a database based on the component name; Determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image; If not, the steps after determining the component name of the target to be aligned based on the visible light image and the hyperspectral image are performed.

[0006] In a possible implementation, judging whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image includes: Acquire a first virtual model of a component corresponding to the standard hyperspectral image in a database, and construct a second virtual model of the target to be aligned based on the visible light image and the point cloud data information; Performing posture adjustment on the second virtual model so that the posture of the second virtual model is consistent with the posture of the first virtual model, and recording posture adjustment parameter information in the process of performing posture adjustment on the second virtual model; Performing attitude adjustment on the hyperspectral image based on the attitude adjustment parameter information; A first spatial rectangular coordinate system is constructed with the center of gravity of the standard hyperspectral image as the origin, and a second spatial rectangular coordinate system is constructed with the center of gravity of the hyperspectral image after the posture adjustment as the origin; The standard hyperspectral image is segmented in the first spatial rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and the posture-adjusted hyperspectral image is segmented in the second spatial rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks; determining whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks; If they are inconsistent, it is determined that the target to be aligned has defects; If they are consistent, for each of the second spectral image blocks, judging whether the second spectral image block has a defect based on each of the first spectral image blocks; If none of the second spectral image blocks has defects, it is determined that the object to be aligned has no defects; If any of the second spectral image blocks has a defect, it is determined that the object to be aligned has a defect.

[0007] In a possible implementation manner, judging whether the second spectral image block has defects based on each of the first spectral image blocks includes: Determine whether there is a target first spectral image block corresponding to the second spectral image block in each of the first spectral image blocks; wherein a first coordinate point corresponding to the center of gravity of the target first spectral image block in the first spatial rectangular coordinate system is consistent with a second coordinate point corresponding to the second spectral image block in the second spatial rectangular coordinate system; If so, obtaining the similarity between the target first spectral image block and the second spectral image block, and comparing the similarity with a preset similarity; If the similarity is greater than the preset similarity, it is determined that the second spectral image block has no defects; If the similarity is not greater than the preset similarity, it is determined that the second spectral image block has defects.

[0008] In a possible implementation, constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information includes: Constructing a virtual geometric structure of the target to be aligned in the virtual alignment scene based on the point cloud data information; The virtual geometric structure is rendered based on the visible light image to obtain the virtual component.

[0009] In a second aspect, the present application provides an industrial vision automatic alignment device, comprising: An acquisition module, used to acquire multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; A determination module, used to determine the component name of the target to be aligned based on the visible light image and the hyperspectral image; An acquisition module, used for acquiring a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; A construction module, used for constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; A control module, used for controlling the movement of the virtual component to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; The alignment processing module is used to control the manipulator to perform alignment processing on the target to be aligned based on the movement trajectory.

[0010] In a third aspect, the present application provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the industrial vision automatic alignment method as described in any one of the above items is implemented.

[0011] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the industrial vision automatic alignment method as described in any one of the above items is implemented.

[0012] The present application provides an industrial vision automatic alignment method and related equipment, the method comprising: collecting multimodal image information of a target to be aligned by a multimodal image acquisition device at a preset position; wherein the multimodal image information comprises a visible light image, point cloud data information and a hyperspectral image; determining the component name of the target to be aligned based on the visible light image and the hyperspectral image; acquiring a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene comprises a standard virtual component of the target to be aligned; constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component in the process of controlling the movement of the virtual component; and controlling a manipulator to perform alignment processing on the target to be aligned based on the movement trajectory. The method, on the one hand, by determining the component name of the target to be aligned based on the visible light image and the hyperspectral image, can accurately distinguish components with similar appearance but different chemical compositions, which helps to prevent confusion between components with similar appearance but different chemical compositions, thereby improving the accuracy and precision of the alignment results; on the other hand, by controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component, an accurate simulation of the alignment process of the target to be aligned is achieved, which helps to improve the accuracy during the actual alignment operation and achieve high-precision alignment of small and complex components. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0014] Figure 1 A schematic diagram of the process flow of the industrial vision automatic alignment method provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of an industrial vision automatic alignment device provided in an embodiment of the present application; Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0017] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0018] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0019] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] See also Figure 1 , Figure 1 A flow chart of the industrial vision automatic alignment method provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the industrial vision automatic alignment method provided in the embodiment of the present application includes steps S1 to S6.

[0021] Step S1, collecting multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images.

[0022] Among them, the multimodal image acquisition device includes multiple multimodal image acquisition units, each of which is arranged at a different position, and each of the multimodal image acquisition units respectively acquires multimodal images of different parts of the target to be aligned to obtain the multimodal image information, and the multimodal image information includes a global multimodal image of the target to be aligned, and each of the multimodal image acquisition units includes a camera, a laser radar and a hyperspectral camera, wherein the camera is used to acquire visible light images, the laser radar is used to acquire point cloud data information, and the hyperspectral camera is used to acquire hyperspectral images.

[0023] Step S2: determining the component name of the target to be aligned based on the visible light image and the hyperspectral image.

[0024] Specifically, the visible light image and the hyperspectral image are input into a preset component recognition model to obtain the component name of the target to be aligned, and the component recognition model is a pre-trained neural network model.

[0025] It can be understood that the hyperspectral image helps to identify the chemical composition information of the target to be aligned. Step S2 determines the component name of the target to be aligned based on the visible light image and the hyperspectral image, which can accurately distinguish components with similar appearance but different chemical composition, help prevent confusion between components with similar appearance but different chemical composition, and thus improve the accuracy and precision of the alignment results.

[0026] Step S3: acquiring a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned.

[0027] Among them, the database stores the correspondence between component names and virtual alignment scenes, and the method for obtaining the virtual alignment scene of the target to be aligned is to accurately align the component consistent with the target to be aligned in the alignment scene corresponding to the multimodal image acquisition device, obtain standard multimodal image information of the alignment scene through the multimodal image acquisition device, and construct the virtual alignment scene based on the standard visible light image and standard point cloud data information in the standard multimodal image information.

[0028] Step S4: constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information.

[0029] Specifically, step S4 includes the following steps: Based on the point cloud data information, a virtual geometric structure of the target to be aligned is constructed in the virtual alignment scene; specifically, for each point in the point cloud data information, each point is mapped to the virtual alignment scene based on the position of the point in the alignment scene to obtain the virtual geometric structure of the target to be aligned; The virtual geometric structure is rendered based on the visible light image to obtain the virtual component.

[0030] Step S5: Control the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and record the movement trajectory of the virtual component during the process of controlling the movement of the virtual component.

[0031] Step S6: controlling the robot to perform alignment processing on the target to be aligned based on the movement trajectory.

[0032] The method provided in this embodiment, on the one hand, by determining the component name of the target to be aligned based on the visible light image and the hyperspectral image, can accurately distinguish components with similar appearance but different chemical compositions, which helps to prevent confusion between components with similar appearance but different chemical compositions, thereby improving the accuracy and precision of the alignment results; on the other hand, by controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component, an accurate simulation of the alignment process of the target to be aligned is achieved, which helps to improve the accuracy during the actual alignment operation and achieve high-precision alignment of small and complex components.

[0033] In some embodiments, before acquiring the virtual alignment scene of the target to be aligned in the database based on the component name, the method further comprises the following steps: Acquire a standard hyperspectral image of the target to be aligned in a database based on the component name; Determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image; If not, the steps after determining the component name of the target to be aligned based on the visible light image and the hyperspectral image are performed.

[0034] In this embodiment, judging whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image includes the following steps: A first virtual model of a component corresponding to the standard hyperspectral image is obtained in a database, and a second virtual model of the target to be aligned is constructed based on the visible light image and the point cloud data information; it can be understood that the component and the target to be aligned are of the same type, the component has no defects, and the first virtual model is a virtual model consistent with the component constructed based on the posture of the component when the standard hyperspectral image of the component is obtained; Performing posture adjustment on the second virtual model so that the posture of the second virtual model is consistent with the posture of the first virtual model, and recording posture adjustment parameter information in the process of adjusting the posture of the second virtual model; wherein the posture adjustment parameter information includes a rotation direction and a rotation angle corresponding to the rotation direction, and the rotation direction includes at least one, when adjusting the posture of the second virtual model, the second virtual model is rotated with the central axis of the second virtual model as the rotation axis; Performing attitude adjustment on the hyperspectral image based on the attitude adjustment parameter information; A first spatial rectangular coordinate system is constructed with the center of gravity of the standard hyperspectral image as the origin, and a second spatial rectangular coordinate system is constructed with the center of gravity of the hyperspectral image after the posture adjustment as the origin; The standard hyperspectral image is segmented in the first spatial rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and the posture-adjusted hyperspectral image is segmented in the second spatial rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks; determining whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks; If they are inconsistent, it is determined that the target to be aligned has defects; If they are consistent, for each of the second spectral image blocks, judging whether the second spectral image block has a defect based on each of the first spectral image blocks; If none of the second spectral image blocks has defects, it is determined that the object to be aligned has no defects; If any of the second spectral image blocks has a defect, it is determined that the object to be aligned has a defect.

[0035] In this embodiment, judging whether the second spectral image block has defects based on each of the first spectral image blocks includes the following steps: Determine whether there is a target first spectral image block corresponding to the second spectral image block in each of the first spectral image blocks; wherein a first coordinate point corresponding to the center of gravity of the target first spectral image block in the first spatial rectangular coordinate system is consistent with a second coordinate point corresponding to the second spectral image block in the second spatial rectangular coordinate system; If so, obtaining the similarity between the target first spectral image block and the second spectral image block, and comparing the similarity with a preset similarity; specifically, inputting the target first spectral image block into a preset spectral image block feature extraction model to obtain a first feature vector, and inputting the second spectral image block into the spectral image block feature extraction model to obtain a second feature vector, and determining the cosine value of the first feature vector and the second feature vector as the similarity; If the similarity is greater than the preset similarity, it is determined that the second spectral image block has no defects; If the similarity is not greater than the preset similarity, it is determined that the second spectral image block has defects.

[0036] It can be understood that the above-mentioned method of judging whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image realizes a refined defect detection method, which helps to improve the accuracy of defect detection results.

[0037] The method provided in this embodiment can pre-detect possible defects of the target to be aligned by comparing the hyperspectral image of the target to be aligned with the standard hyperspectral image before the target is aligned, which helps to effectively prevent defective parts from entering the subsequent alignment process, reduces resource waste and improves the reliability of the overall process.

[0038] See also Figure 2 , Figure 2 A schematic block diagram of the structure of the industrial vision automatic alignment device 100 provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the industrial vision automatic alignment device 100 provided in the embodiment of the present application includes: The acquisition module 110 is used to acquire multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images.

[0039] The determination module 120 is configured to determine the component name of the target to be aligned based on the visible light image and the hyperspectral image.

[0040] The acquisition module 130 is used to acquire the virtual alignment scene of the target to be aligned in the database based on the component name; wherein the virtual alignment scene includes the standard virtual component of the target to be aligned.

[0041] The construction module 140 is used to construct a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information.

[0042] The control module 150 is used to control the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and to record the movement trajectory of the virtual component during the process of controlling the movement of the virtual component.

[0043] The alignment processing module 160 is used to control the robot to perform alignment processing on the target to be aligned based on the movement trajectory.

[0044] It should be noted that technicians in the relevant technical field can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned industrial vision automatic alignment method embodiment, and will not be repeated here.

[0045] The terminal device 200 provided in the above embodiment can be implemented in the form of a computer program. The computer program can be Figure 3 The system is run on the terminal device 200 shown.

[0046] See also Figure 3 , Figure 3 The present invention provides a schematic block diagram of the structure of a terminal device 200 according to an embodiment of the present application. The terminal device 200 includes a processor 201 and a memory 202. The processor 201 and the memory 202 are connected via a device bus 203, wherein the memory 202 may include a non-volatile storage medium and an internal memory.

[0047] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 201, the processor 201 can execute any of the above-mentioned industrial vision automatic alignment methods.

[0048] The processor 201 is used to provide computing and control capabilities to support the operation of the entire terminal device 200.

[0049] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 201, the processor 201 can execute any of the above-mentioned industrial vision automatic alignment methods.

[0050] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal device 200 involved in the scheme of the present application. The specific terminal device 200 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0051] It should be understood that the processor 201 may be a central processing unit (CPU), and the processor 201 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0052] In some embodiments, the processor 201 is used to run a computer program stored in the memory to implement the following steps: Collecting multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; Determining a component name of the target to be aligned based on the visible light image and the hyperspectral image; Acquire a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; Constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; Controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; Based on the movement trajectory, the robot is controlled to perform alignment processing on the target to be aligned.

[0053] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the terminal device 200 described above can refer to the corresponding process of the aforementioned industrial vision automatic alignment method, and will not be repeated here.

[0054] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement the industrial vision automatic alignment method provided in the embodiment of the present application.

[0055] The computer-readable storage medium may be an internal storage unit of the terminal device 200 in the aforementioned embodiment, such as a hard disk or memory of the terminal device 200. The computer-readable storage medium may also be an external storage device of the terminal device 200, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the terminal device 200.

[0056] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An industrial vision automatic alignment method, characterized in that: include: Collecting multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; Determining a component name of the target to be aligned based on the visible light image and the hyperspectral image; Acquire a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; Constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; Controlling the virtual component to move to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; Based on the movement trajectory, the robot is controlled to perform alignment processing on the target to be aligned.

2. The industrial vision automatic alignment method according to claim 1, characterized in that: Before acquiring the virtual alignment scene of the target to be aligned in the database based on the component name, the method further includes: Acquire a standard hyperspectral image of the target to be aligned in a database based on the component name; Determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image; If not, the steps after determining the component name of the target to be aligned based on the visible light image and the hyperspectral image are performed.

3. The industrial vision automatic alignment method according to claim 2 is characterized in that: The determining whether the target to be aligned has defects based on the hyperspectral image and the standard hyperspectral image includes: Acquire a first virtual model of a component corresponding to the standard hyperspectral image in a database, and construct a second virtual model of the target to be aligned based on the visible light image and the point cloud data information; Performing posture adjustment on the second virtual model so that the posture of the second virtual model is consistent with the posture of the first virtual model, and recording posture adjustment parameter information in the process of performing posture adjustment on the second virtual model; Performing attitude adjustment on the hyperspectral image based on the attitude adjustment parameter information; A first spatial rectangular coordinate system is constructed with the center of gravity of the standard hyperspectral image as the origin, and a second spatial rectangular coordinate system is constructed with the center of gravity of the hyperspectral image after the posture adjustment as the origin; The standard hyperspectral image is segmented in the first spatial rectangular coordinate system based on a preset spectral image segmentation method to obtain a plurality of first spectral image blocks, and the posture-adjusted hyperspectral image is segmented in the second spatial rectangular coordinate system based on the spectral image segmentation method to obtain a plurality of second spectral image blocks; determining whether the number of the first spectral image blocks is consistent with the number of the second spectral image blocks; If they are inconsistent, it is determined that the target to be aligned has defects; If they are consistent, for each of the second spectral image blocks, judging whether the second spectral image block has a defect based on each of the first spectral image blocks; If none of the second spectral image blocks has defects, it is determined that the object to be aligned has no defects; If any of the second spectral image blocks has a defect, it is determined that the object to be aligned has a defect.

4. The industrial vision automatic alignment method according to claim 3 is characterized in that: The determining whether the second spectral image block has a defect based on each of the first spectral image blocks includes: Determine whether there is a target first spectral image block corresponding to the second spectral image block in each of the first spectral image blocks; wherein a first coordinate point corresponding to the center of gravity of the target first spectral image block in the first spatial rectangular coordinate system is consistent with a second coordinate point corresponding to the second spectral image block in the second spatial rectangular coordinate system; If so, obtaining the similarity between the target first spectral image block and the second spectral image block, and comparing the similarity with a preset similarity; If the similarity is greater than the preset similarity, it is determined that the second spectral image block has no defects; If the similarity is not greater than the preset similarity, it is determined that the second spectral image block has defects.

5. The industrial vision automatic alignment method according to claim 1, characterized in that: The step of constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information includes: Constructing a virtual geometric structure of the target to be aligned in the virtual alignment scene based on the point cloud data information; The virtual geometric structure is rendered based on the visible light image to obtain the virtual component.

6. An industrial vision automatic alignment device, characterized in that: include: An acquisition module, used to acquire multimodal image information of the target to be aligned through a multimodal image acquisition device at a preset position; wherein the multimodal image information includes visible light images, point cloud data information and hyperspectral images; A determination module, used to determine the component name of the target to be aligned based on the visible light image and the hyperspectral image; An acquisition module, used for acquiring a virtual alignment scene of the target to be aligned in a database based on the component name; wherein the virtual alignment scene includes a standard virtual component of the target to be aligned; A construction module, used for constructing a virtual component of the target to be aligned in the virtual alignment scene based on the visible light image and the point cloud data information; A control module, used for controlling the movement of the virtual component to overlap with the standard virtual component in the virtual alignment scene, and recording the movement trajectory of the virtual component during the process of controlling the movement of the virtual component; The alignment processing module is used to control the manipulator to perform alignment processing on the target to be aligned based on the movement trajectory.

7. A terminal device, characterized in that: The terminal device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the industrial vision automatic alignment method as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the industrial vision automatic alignment method according to any one of claims 1 to 5 is implemented.

Images