Real-time industrial visual inspection processing method and related device

Abstract

The invention discloses a real-time industrial visual inspection processing method and a related device, and relates to the technical field of machine vision, and the method comprises the following steps: constructing a context scheduling architecture based on a language interface; collecting image data based on an industrial camera, and preprocessing the image data based on the context scheduling architecture to obtain preprocessed image data; performing visual detection on the preprocessed image data by adopting a CANN and operator fusion-based target detection model based on an AI core to obtain a visual detection tensor; based on an AscendCL interface, the visual detection tensor is utilized to carry out result analysis of parallel calculation, and result analysis data is obtained; and generating a control instruction based on the result analysis data, and executing the control instruction based on the industrial production line. According to the method, the problems of high end-to-end processing delay and poor real-time performance caused by long data flow path, loose software and hardware coupling and single optimization dimension can be solved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a real-time industrial vision inspection and processing method and related apparatus. Background Technology

[0002] Industrial visual inspection, as a key link in ensuring product quality in modern manufacturing, has gradually evolved from inefficient manual inspection and traditional algorithms with insufficient robustness to intelligent inspection centered on deep learning. Although artificial intelligence technology has improved inspection accuracy, most current mainstream solutions are deployed on general-purpose computing platforms such as central processing units and graphics processing units. When applied to industrial scenarios with stringent real-time requirements, their inherent architectural limitations expose new performance bottlenecks.

[0003] Specifically, a complete inspection task flow is fragmented across different hardware units. Steps such as image preprocessing, model inference, and post-processing result in frequent data copying between the host and accelerator, generating significant data transfer overhead. This loosely coupled hardware and software model not only causes difficult-to-optimize end-to-end processing latency but also increases the complexity for developers switching between different toolchains. Ultimately, this prevents the overall system processing speed from meeting the stringent requirements of processing tens or even hundreds of products per second on high-speed production lines. Therefore, there is an urgent need for a new approach based on deep hardware and software collaboration that can uniformly optimize the entire data processing flow to overcome the performance limitations of existing technologies in industrial real-time visual inspection applications. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a real-time industrial visual inspection and processing method and related device, which can solve the problems of high end-to-end processing latency and poor real-time performance caused by long data flow paths, loose hardware and software coupling, and single optimization dimension.

[0005] To address the aforementioned technical problems, this invention provides a real-time industrial visual inspection and processing method, the method comprising: Context scheduling architecture is built based on language interfaces; Image data is acquired using an industrial camera, and preprocessed based on the aforementioned context scheduling architecture to obtain preprocessed image data. Based on the core of artificial intelligence (AI), a target detection model based on the neural network heterogeneous computing architecture CANN and operator fusion is used to perform visual detection on preprocessed image data to obtain the visual detection tensor. Based on the AscendCL interface, the results of parallel computation using the visual detection tensor are analyzed to obtain result analysis data; Based on the results, analysis data is used to generate control commands, which are then executed on the industrial production line.

[0006] Optionally, the context scheduling architecture built based on the language interface includes: The computational language interface is invoked, and the system resources of the AI ​​processor are initialized based on the computational language interface to obtain the initialized system resources; A unified runtime management context and data flow are constructed based on the initialization system resources, and a context scheduling architecture is determined based on the runtime management context and data flow.

[0007] Optionally, the preprocessing of image data based on the context scheduling architecture to obtain preprocessed image data includes: The digital vision preprocessing hardware unit is invoked based on the aforementioned context scheduling architecture, and hardware tasks are generated based on the aforementioned digital vision preprocessing hardware unit. Based on the hardware task, the image data is decoded, scaled, converted in color gamut, cropped and normalized to obtain preprocessed image data.

[0008] Optionally, the target detection model based on the AI ​​core and employing a neural network heterogeneous computing architecture CANN and operator fusion performs visual detection on the preprocessed image data to obtain a visual detection tensor, including: Obtain a sample image dataset and train a deep neural network based on the sample image dataset to obtain a trained deep neural network, and use the trained deep neural network as the initial detection model. Obtain the initial computation graph of the initial detection model, and optimize the initial computation graph to obtain an optimized computation graph; Based on the optimized computation graph, an operator connection graph is determined, and based on the operator connection graph and a preset objective function, a target operator fusion strategy is determined. Based on the operator fusion strategy and CANN, the initial detection model is subjected to operator fusion and compilation optimization to obtain the target detection model; The preprocessed image data is input into the AI ​​core object detection model for visual detection to obtain the visual detection tensor.

[0009] Optionally, determining the target operator fusion strategy based on the operator connectivity graph and a preset objective function includes: Based on preset fusion rules, several operator fusion groups are obtained using the operator connection graph; Several initial operator fusion strategies are determined based on the aforementioned operator fusion group; The complexity of each initial operator fusion strategy is calculated based on a preset objective function to obtain the complexity of each initial operator fusion strategy. The initial operator fusion strategy with the lowest complexity is taken as the target operator fusion strategy.

[0010] Optionally, the result analysis of the parallel computation using the visual detection tensor based on the AscendCL interface to obtain result analysis data includes: Device resources are obtained based on the AscendCL interface, and several target threads for parallel computing are established based on the device resources. Based on the parallel processing and data simplification of each target thread using the visual detection tensor for defect judgment, coordinate parsing and logical filtering, result analysis data is obtained.

[0011] Optionally, the step of generating control instructions based on the analyzed data and executing the control instructions based on the industrial production line includes: Based on the results, the data is analyzed and control instructions are generated by copying data into memory, and then the control instructions are transmitted to the programmable logic controller in the industrial production line. The programmable logic controller controls the corresponding mechanism of the industrial production line to execute the control commands.

[0012] In addition, the present invention also provides a real-time industrial visual inspection and processing device, the device comprising: Architecture building block: Used to build context scheduling architecture based on language interfaces; Image preprocessing module: used to acquire image data based on an industrial camera and preprocess the image data based on the context scheduling architecture to obtain preprocessed image data; Visual inspection module: Used to perform visual inspection on preprocessed image data based on the AI ​​core and a target detection model based on the neural network heterogeneous computing architecture CANN and operator fusion, to obtain the visual inspection tensor; Results Analysis Module: Used for analyzing the results of parallel computation using the visual detection tensor based on the AscendCL interface, and obtaining results analysis data; Instruction generation and execution module: used to generate control instructions based on the result analysis data, and execute the control instructions based on the industrial production line.

[0013] In addition, the present invention also provides an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the above-described real-time industrial vision inspection processing method.

[0014] In addition, the present invention also provides a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the above-described real-time industrial vision inspection processing method.

[0015] In this embodiment of the invention, a context scheduling architecture is constructed based on a language interface. Image data is preprocessed using this architecture, eliminating the need for frequent data transfer and significantly reducing preprocessing time. Based on the AI ​​core, a target detection model using CANN and operator fusion is employed to perform visual detection on the preprocessed image data, reducing the number of core scheduling operations and improving the efficiency of visual detection. The results of parallel computation using the AscendCL interface are analyzed, and control commands are generated based on the analyzed data. These control commands are then executed based on the industrial production line. Combined with parallel computation using the AscendCL interface, this ensures real-time end-to-end processing and solves the problems of high latency and poor real-time performance caused by long data flow paths, loose hardware-software coupling, and single optimization dimensions. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the real-time industrial vision inspection and processing method in an embodiment of the present invention. Figure 2 This is a flowchart illustrating a real-time industrial vision inspection and processing method according to another embodiment of the present invention. Figure 3 This is a schematic diagram of the structural composition of the real-time industrial vision inspection and processing device in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating the real-time industrial vision inspection processing method according to an embodiment of the present invention. The method includes: S11: Building a context scheduling architecture based on language interfaces; In the specific implementation of this invention, a computational language interface is invoked, and the system resources of the AI ​​processor are initialized based on the computational language interface to obtain initialized system resources; a unified operation management context and data flow are constructed based on the initialized system resources, and a context scheduling architecture is determined based on the operation management context and data flow. Subsequent steps are executed in this context scheduling architecture, which can shorten the data flow path and avoid frequent data copying between the central processing unit and the accelerator.

[0020] S12: Acquire image data based on an industrial camera, and preprocess the image data based on the context scheduling architecture to obtain preprocessed image data; In the specific implementation of this invention, image data is acquired based on an industrial camera, and a digital vision preprocessing hardware unit is invoked based on a context scheduling architecture. A hardware task is generated based on the digital vision preprocessing hardware unit. Based on the hardware task, the image data is decoded, scaled, converted in color gamut, cropped, and normalized to obtain preprocessed image data. The digital vision preprocessing hardware unit greatly compresses the preprocessing time, thereby significantly reducing the end-to-end latency from image input to instruction output.

[0021] S13: Based on the core of artificial intelligence (AI), a target detection model based on the neural network heterogeneous computing architecture CANN and operator fusion is used to perform visual detection on the preprocessed image data to obtain the visual detection tensor. In the specific implementation of this invention, a sample image dataset is acquired, and a deep neural network is trained based on the sample image dataset to obtain a trained deep neural network, which is then used as the initial detection model. The initial computation graph of the initial detection model is acquired and optimized to obtain an optimized computation graph. An operator connection graph is determined based on the optimized computation graph, and a target operator fusion strategy is determined based on the operator connection graph and a preset objective function. Based on the operator fusion strategy and a Compute Architecture for Neural Networks (CANN), the initial detection model undergoes operator fusion and compilation optimization processing to obtain a target detection model. The preprocessed image data is input into the target detection model of the Artificial Intelligence (AI) core for visual detection, obtaining a visual detection tensor. This allows data to flow rapidly directly within the Ascend AI core, significantly reducing memory bandwidth overhead and improving the overall inference throughput of the model.

[0022] S14: Analyze the results of parallel computation using the visual detection tensor based on the AscendCL interface to obtain result analysis data; In the specific implementation of this invention, Device resources are obtained based on the Ascend Computing Language (AscendCL) interface, and several target threads for parallel computing are established based on the Device resources. Based on the visual inspection tensor of each target thread, parallel processing and data simplification processing are performed for defect judgment, coordinate analysis and logical filtering to obtain result analysis data. This avoids the huge cross-bus transmission delay caused by copying large tensor data back to the host central processing unit for post-processing, and ensures the real-time performance of end-to-end processing.

[0023] S15: Generate control instructions based on the analyzed data, and execute the control instructions based on the industrial production line.

[0024] In the specific implementation of this invention, control instructions are generated by copying the result analysis data into memory and transmitted to the programmable logic controller (PLC) of the industrial production line. The PLC controls the corresponding mechanism of the industrial production line to execute the control instructions, ensuring the real-time processing of the industrial production line.

[0025] In this embodiment of the invention, a context scheduling architecture is constructed based on a language interface. Image data is preprocessed using this architecture, eliminating the need for frequent data transfer and significantly reducing preprocessing time. Based on the AI ​​core, a target detection model using CANN and operator fusion is employed to perform visual detection on the preprocessed image data, reducing the number of core scheduling operations and improving the efficiency of visual detection. The results of parallel computation using the AscendCL interface are analyzed, and control commands are generated based on the analyzed data. These control commands are then executed based on the industrial production line. Combined with parallel computation using the AscendCL interface, this ensures real-time end-to-end processing and solves the problems of high latency and poor real-time performance caused by long data flow paths, loose hardware-software coupling, and single optimization dimensions.

[0026] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating a real-time industrial vision inspection processing method according to another embodiment of the present invention, the method comprising: S201: Building a context scheduling architecture based on language interfaces; In a specific implementation of this invention, the construction of a context scheduling architecture based on a language interface includes: calling a computational language interface and initializing the system resources of the AI ​​processor based on the computational language interface to obtain initialized system resources; constructing a unified runtime management context and data flow based on the initialized system resources; and determining the context scheduling architecture based on the runtime management context and data flow.

[0027] Specifically, the computational language interface is invoked, and the system resources of the AI ​​processor, which is the Ascend AI processor, are initialized based on the computational language interface to obtain the initialized system resources.

[0028] Based on the initialization system resources, a unified operation management context and data flow are constructed, and a context scheduling architecture is determined based on the operation management context and data flow. Subsequent steps are all executed in this context scheduling architecture, placing all core computing processes within the unified hardware and software context architecture of the Ascend AI processor. This minimizes the data flow path and avoids frequent data copying between the central processing unit and the accelerator.

[0029] S202: Acquire image data based on an industrial camera, and preprocess the image data based on the context scheduling architecture to obtain preprocessed image data; In a specific implementation of the present invention, the step of preprocessing the image data based on the context scheduling architecture to obtain preprocessed image data includes: calling a digital vision preprocessing hardware unit based on the context scheduling architecture, and generating a hardware task based on the digital vision preprocessing hardware unit; and performing decoding, scaling, color gamut conversion, cropping, and normalization processing on the image data based on the hardware task to obtain preprocessed image data.

[0030] Specifically, image data of a target product, such as a circuit board, is acquired using an industrial camera. The digital vision preprocessing hardware unit is invoked based on the context scheduling architecture; that is, the built-in digital vision preprocessing hardware unit is directly invoked from the Ascend AI processor's device memory via the context scheduling architecture. A hardware task is generated based on this digital vision preprocessing hardware unit, which encapsulates multiple steps of image preprocessing into a unified hardware task.

[0031] Based on the hardware task, the image data undergoes decoding, scaling, color gamut conversion, cropping, and normalization. This preprocessing is performed within the encapsulated software task. Image data decoding can employ entropy decoding, converting the image file into a directly processable uncompressed bitmap format for application or system processing. Scaling unifies the input size, color gamut conversion maintains color consistency and accuracy across different color spaces, cropping removes redundant information, and normalization converts pixel values ​​to a uniform range, improving data processing efficiency and model performance. After these processes, image data preprocessing is complete, yielding preprocessed image data. Preprocessing image data within the hardware task encapsulated by the character vision preprocessing hardware unit ensures that data resides entirely in device memory from reception to processing, eliminating frequent data transfers via the host CPU and high-speed serial computer expansion bus. Compared to the fragmented cross-bus / cross-device processing flow of conventional CPU plus GPU solutions, this significantly eliminates data transfer overhead and scheduling latency, achieving true process integration and zero-copy at the device end, which is crucial for ensuring the real-time performance of industrial vision.

[0032] S203: Obtain a sample image dataset, and train a deep neural network based on the sample image dataset to obtain a trained deep neural network, and use the trained deep neural network as the initial detection model. In the specific implementation of this invention, a sample image dataset is obtained, which is sample image data of industrial products. A deep neural network is trained based on the sample image dataset to obtain a trained deep neural network. The trained deep neural network is used as the initial detection model. The deep neural network includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

[0033] S204: Obtain the initial computation graph of the initial detection model, and optimize the initial computation graph to obtain an optimized computation graph; In the specific implementation of this invention, an artificial intelligence framework for the initial detection model is determined, and a corresponding initial computation graph is determined based on the artificial intelligence framework. The initial computation graph includes nodes and directed edges. Each node is an operator, such as a convolution operator and a pooling operator. Directed edges represent the data transfer relationship between nodes. A preset inference acceleration engine is used to optimize the initial computation graph. The inference acceleration engine reads the initial computation graph through multi-level intermediate representation graph compilation technology. Multi-level intermediate representation graph compilation technology is mainly used to convert the computation graph into executable code or optimized hardware instructions. Its core is to analyze, optimize, and transform the graph structure through multi-level intermediate representation capabilities, and perform hardware-independent optimizations such as shape inference and constant propagation, as well as hardware-related optimizations such as software pipelining, to obtain an optimized computation graph.

[0034] S205: Determine the operator connection graph based on the optimized computation graph, and determine the target operator fusion strategy based on the operator connection graph and a preset objective function; In a specific implementation of this invention, determining the target operator fusion strategy based on the operator connection graph and a preset objective function includes: obtaining several operator fusion groups using the operator connection graph based on preset fusion rules; determining several initial operator fusion strategies based on the operator fusion groups; calculating the complexity of each initial operator fusion strategy based on the preset objective function to obtain the complexity corresponding to each initial operator fusion strategy; and selecting the initial operator fusion strategy with the lowest complexity as the target operator fusion strategy.

[0035] Specifically, an operator connection graph is determined based on the optimized computation graph, that is, the connections between operators are determined according to the nodes and directed edges of the optimized computation graph, forming an operator connection graph. Several operator fusion groups are obtained using the operator connection graph based on preset fusion rules. The preset fusion rules include operator type, operator connection method, operator computational complexity, and operator dependency relationship. Operator types include convolution, pooling, normalization, etc.; operator connection methods include multiple-input multiple-output, single-input multiple-output, etc.; operator computational complexity is calculated by calculating the time and space complexity of the operator; and operators with operator dependencies in the operator connection graph are fused according to the preset fusion rules to form several operator fusion groups. Adjacent operators with data dependencies can be fused, such as fusion of convolution operators and batch normalization operators.

[0036] Based on the operator fusion groups, several initial operator fusion strategies are determined. All operator fusion groups are then arranged and combined to obtain several initial operator fusion strategies. The complexity of each initial operator fusion strategy is calculated based on a preset objective function, which is the sum of time and space complexity. The time and space complexity of the fused operator groups and the unfused operators in each initial operator fusion strategy are calculated, and the sum of these two complexities yields the complexity of each initial operator fusion strategy. The initial operator fusion strategy with the lowest complexity is selected as the target operator fusion strategy.

[0037] S206: Based on the operator fusion strategy and CANN, the initial detection model is subjected to operator fusion and compilation optimization to obtain the target detection model; In the specific implementation of this invention, the operator fusion strategy is input into the CANN tool, fusing multiple small operators in the initial detection model into a high-performance "garlic bulb" (or "garlic bulb"), reducing the interaction overhead of the computing core. CANN is an architecture specifically designed and optimized for high-performance neural network computing needs. At the hardware level, the Ascend AI processor features customized computing resource design and deep functional adaptation, providing a powerful hardware foundation for improving neural network computing performance. At the software level, the software stack included in CANN provides functions for managing network models, computational flows, and data flows, supporting the execution flow of the neural network on heterogeneous processors. After the initial detection model is compiled and optimized through CANN, the final target detection model is obtained. Through operator fusion and compilation optimization, the write and read operations of intermediate tensors on the on-chip device memory are eliminated, allowing data to flow directly and quickly in the Ascend AI core's ultra-high-speed cache / register, thereby significantly reducing memory bandwidth overhead, reducing the number of computing core scheduling operations, achieving the ultimate efficient utilization of the computing resources of the Da Vinci architecture, and improving the overall inference throughput.

[0038] S207: Input the preprocessed image data into the AI ​​core object detection model for visual detection to obtain the visual detection tensor; In the specific implementation of this invention, the preprocessed image data is input into the target detection model of the AI ​​core for visual detection, and a tensor of the visual detection result is obtained. The visual detection result, such as defect type and defect location, is the visual detection tensor.

[0039] S208: Analyze the results of parallel computation using the visual detection tensor based on the AscendCL interface to obtain result analysis data; In the specific implementation of this invention, the analysis of the results of parallel computation using the visual inspection tensor based on the AscendCL interface to obtain result analysis data includes: obtaining Device resources based on the AscendCL interface, and establishing several target threads for parallel computation based on the Device resources; and performing parallel processing and data simplification processing of defect judgment, coordinate parsing and logical filtering using the visual inspection tensor based on each target thread to obtain result analysis data.

[0040] Specifically, the visual detection tensor output by the model inference is not immediately returned to the host CPU, but continues to be processed in the device memory of the Ascend AI processor. Device resources are obtained based on the AscendCL interface. The AscendCL interface is the Ascend Computing Open Programming Framework, which encapsulates the underlying Ascend Computing service interfaces. It provides API libraries for runtime resource management, model loading and execution, operator loading and execution, image data encoding / decoding / cropping / scaling, etc., enabling deep learning inference computation, image preprocessing, and single-operator accelerated computation on the Ascend CANN platform. Simply put, it's a unified API framework that allows access to all resources. Device resources are used to describe, manage, and access device resources, and several target threads for parallel computation are established based on these Device resources. Target threads corresponding to subsequent defect judgment, coordinate parsing, and logical filtering are established according to the obtained Device resources, and each target thread performs parallel computation.

[0041] Based on the parallel processing and data simplification of defect judgment, coordinate parsing and logical filtering using the visual detection tensor by each target thread, the defect judgment, coordinate parsing and logical filtering are executed in parallel according to the corresponding target thread, and the resulting data is simplified to obtain result analysis data, such as the presence, coordinates and types of defects. The defect judgment, coordinate parsing and logical filtering are downloaded to the AI ​​core threads in the Ascend AI processor for execution. The entire process is completed in the device memory, thereby avoiding the huge cross-bus transmission delay caused by copying large result tensors back to the host central processing unit for post-processing, and ensuring the real-time performance of end-to-end processing. The complete visual inspection computation task flow, including image preprocessing, model inference, and post-processing, is treated as an indivisible whole and executed on the Ascend AI processor. Following the principle of minimal data interaction, the host is only responsible for initiating the task and receiving a final decision result with a data volume much smaller than the original image. All intermediate data generated during task execution forms a closed loop within the device's memory and does not exchange with the host. Multiple processing steps can be efficiently parallelized, significantly increasing the number of products inspected per unit time and meeting the stringent industrial production pace.

[0042] S209: Generate control instructions based on the analyzed data, and execute the control instructions based on the industrial production line.

[0043] In a specific implementation of this invention, the step of generating control instructions based on the result analysis data and executing the control instructions based on the industrial production line includes: generating control instructions by copying the result analysis data into memory and transmitting the control instructions to the programmable logic controller (PLC) of the industrial production line; the PLC controlling the corresponding mechanism of the industrial production line to execute the control instructions.

[0044] Specifically, based on the analyzed results, control instructions are generated using memory copying. This involves copying the analyzed results from the device memory back to the host memory, generating control instructions based on this data, and then transmitting these control instructions to the programmable logic controller (PLC) of the industrial production line. The PLC then controls the corresponding mechanism on the industrial production line to execute the control instructions.

[0045] In this embodiment of the invention, a context scheduling architecture is constructed based on a language interface. Image data is preprocessed using this architecture, eliminating the need for frequent data transfer and significantly reducing preprocessing time. Based on the AI ​​core, a target detection model using CANN and operator fusion is employed to perform visual detection on the preprocessed image data, reducing the number of core scheduling operations and improving the efficiency of visual detection. The results of parallel computation using the AscendCL interface are analyzed, and control commands are generated based on the analyzed data. These control commands are then executed based on the industrial production line. Combined with parallel computation using the AscendCL interface, this ensures real-time end-to-end processing and solves the problems of high latency and poor real-time performance caused by long data flow paths, loose hardware-software coupling, and single optimization dimensions.

[0046] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of the real-time industrial vision inspection and processing device in an embodiment of the present invention. The device includes: Architecture building module 31: Used to build context scheduling architecture based on language interface; Image preprocessing module 32: used to acquire image data based on an industrial camera and preprocess the image data based on the context scheduling architecture to obtain preprocessed image data; Visual inspection module 33: Used to perform visual inspection on preprocessed image data using a target detection model based on the AI ​​core and the CANN neural network heterogeneous computing architecture and operator fusion, and obtain the visual inspection tensor; Result Analysis Module 34: Used for result analysis of parallel computation using the visual detection tensor based on the AscendCL interface, and to obtain result analysis data; Instruction generation and execution module 35: used to generate control instructions based on the result analysis data, and execute the control instructions based on the industrial production line.

[0047] In the specific implementation of this invention, the specific implementation of the device item can be referred to the implementation of the method item above, and will not be repeated here.

[0048] In this embodiment of the invention, a context scheduling architecture is constructed based on a language interface. Image data is preprocessed using this architecture, eliminating the need for frequent data transfer and significantly reducing preprocessing time. Based on the AI ​​core, a target detection model using CANN and operator fusion is employed to perform visual detection on the preprocessed image data, reducing the number of core scheduling operations and improving the efficiency of visual detection. The results of parallel computation using the AscendCL interface are analyzed, and control commands are generated based on the analyzed data. These control commands are then executed based on the industrial production line. Combined with parallel computation using the AscendCL interface, this ensures real-time end-to-end processing and solves the problems of high latency and poor real-time performance caused by long data flow paths, loose hardware-software coupling, and single optimization dimensions.

[0049] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the real-time industrial vision inspection processing method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0050] Example 4 Please see Figure 4 , Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0051] This invention also provides an electronic device, such as... Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. Those skilled in the art will understand that... Figure 4 The illustrated electronic device does not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 41 can be used to store computer program 42 and various functional modules. Processor 43 runs the computer program 42 stored in memory 41, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. Processor 43 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or a processor 43, or any conventional processor, etc. The processors and memories disclosed in this invention include, but are not limited to, these types of processors and memories. The processors and memories disclosed in this invention are merely examples and not intended to be limiting.

[0052] As one embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to perform the real-time industrial vision inspection processing method in any of the above embodiments. For specific implementation processes, please refer to the above embodiments, which will not be repeated here.

[0053] In this embodiment of the invention, a context scheduling architecture is constructed based on a language interface. Image data is preprocessed using this architecture, eliminating the need for frequent data transfer and significantly reducing preprocessing time. Based on the AI ​​core, a target detection model using CANN and operator fusion is employed to perform visual detection on the preprocessed image data, reducing the number of core scheduling operations and improving the efficiency of visual detection. The results of parallel computation using the AscendCL interface are analyzed, and control commands are generated based on the analyzed data. These control commands are then executed based on the industrial production line. Combined with parallel computation using the AscendCL interface, this ensures real-time end-to-end processing and solves the problems of high latency and poor real-time performance caused by long data flow paths, loose hardware-software coupling, and single optimization dimensions.

[0054] Furthermore, the above provides a detailed description of the real-time industrial visual inspection processing method and related apparatus provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A real-time industrial vision inspection processing method, characterized in that, The method comprises: building a context scheduling architecture based on a language interface; acquiring image data based on an industrial camera, and preprocessing the image data based on the context scheduling architecture to obtain preprocessed image data; performing visual detection on the preprocessed image data based on an artificial intelligence (AI) core using a target detection model based on a neural network heterogeneous computing architecture (CANN) and operator fusion to obtain a visual detection tensor; performing result analysis on the visual detection tensor based on an AscendCL interface to obtain result analysis data; generating a control instruction based on the result analysis data, and executing the control instruction based on an industrial production line.

2. The real-time industrial vision inspection processing method of claim 1, wherein, The building of the context scheduling architecture based on the language interface comprises: calling a computing language interface, and initializing system resources of an AI processor based on the computing language interface to obtain initialized system resources; building a unified running management context and a data flow based on the initialized system resources, and determining a context scheduling architecture based on the running management context and the data flow.

3. The real-time industrial vision inspection processing method of claim 1, wherein, The preprocessing of the image data based on the context scheduling architecture to obtain preprocessed image data comprises: calling a digital visual preprocessing hardware unit based on the context scheduling architecture, and generating a hardware task based on the digital visual preprocessing hardware unit; performing decoding, scaling, color gamut conversion, cropping, and normalization processing on the image data based on the hardware task to obtain preprocessed image data.

4. The real-time industrial vision inspection processing method of claim 1, wherein, The visual detection on the preprocessed image data based on the AI core using the target detection model based on the CANN and the operator fusion to obtain the visual detection tensor comprises: obtaining a sample image data set, training a deep neural network based on the sample image data set to obtain a trained deep neural network, and taking the trained deep neural network as an initial detection model; obtaining an initial computation graph of the initial detection model, and optimizing the initial computation graph to obtain an optimized computation graph; determining an operator connection graph based on the optimized computation graph, and determining a target operator fusion strategy based on the operator connection graph in combination with a preset target function; performing operator fusion and compilation optimization processing on the initial detection model based on the operator fusion strategy and the CANN to obtain a target detection model; inputting the preprocessed image data into the target detection model of the AI core for visual detection to obtain a visual detection tensor.

5. The real-time industrial vision inspection processing method of claim 4, wherein, The determination of the target operator fusion strategy based on the operator connection graph in combination with the preset target function comprises: obtaining a plurality of operator fusion groups based on the operator connection graph based on a preset fusion rule; determining a plurality of initial operator fusion strategies based on the operator fusion groups; performing complexity calculation on each initial operator fusion strategy based on the preset target function to obtain a complexity corresponding to each initial operator fusion strategy; taking an initial operator fusion strategy with the minimum complexity as the target operator fusion strategy.

6. The real-time industrial vision inspection processing method of claim 1, wherein, The result analysis based on the AscendCL interface on the visual detection tensor based on parallelization calculation to obtain result analysis data comprises: obtaining a result analysis data based on the result analysis data by using memory copy to generate a control instruction, and transmitting the control instruction to a programmable logic controller of the industrial production line; generating a control instruction based on the result analysis data, and executing the control instruction based on the industrial production line, including:

7. The real-time industrial vision inspection processing method of claim 1, wherein, obtaining a result analysis data based on the result analysis data by using memory copy to generate a control instruction, and transmitting the control instruction to a programmable logic controller of the industrial production line; the programmable logic controller controls the corresponding mechanism of the industrial production line to execute the control instruction. The device includes:

8. A real-time industrial vision inspection processing apparatus, characterized by, an architecture construction module for constructing a context scheduling architecture based on a language interface; an image preprocessing module for collecting image data based on an industrial camera, and preprocessing the image data based on the context scheduling architecture to obtain preprocessed image data; a visual detection module for performing visual detection on the preprocessed image data based on a neural network heterogeneous computing architecture (CANN) and an operator fusion target detection model based on an artificial intelligence (AI) core to obtain a visual detection tensor; a result analysis module for performing result analysis based on the visual detection tensor by using parallel computing based on an AscendCL interface to obtain result analysis data; an instruction generation and execution module for generating a control instruction based on the result analysis data, and executing the control instruction based on the industrial production line. The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the real-time industrial visual detection processing method in any one of claims 1 to 7.

9. An electronic device comprising a processor and a memory, the electronic device comprising: The computer readable storage medium stores computer instructions, when the computer instructions run on the electronic device, make the electronic device execute the real-time industrial visual detection processing method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, ​

Images