A multi-camera cooperative detection method and system based on hierarchical knowledge reasoning

Abstract

The present application relates to the technical field of intelligent manufacturing and machine vision, and provides a multi-camera cooperative detection method and system based on hierarchical knowledge reasoning, which comprises the following steps: constructing a hierarchical knowledge graph; constructing a multi-camera spatial relationship model; constructing a knowledge reasoning rule library and a dynamic knowledge updating mechanism; calling the knowledge reasoning rule library and the dynamic knowledge updating mechanism to update the hierarchical knowledge graph through knowledge reasoning; and controlling the actions of multiple cameras according to the multi-camera spatial relationship model and the updated hierarchical knowledge graph to complete the detection task of a workpiece to be detected. The present application can execute different control strategies for each detection level of different detection processes of a workpiece, automatically and efficiently acquire and process a large amount of new knowledge data, mine potential regular features, adapt to different detection scenes and requirements, and realize omnidirectional visual detection of large-size and complex-structure industrial products such as new energy batteries.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing and machine vision, in particular to a multi-camera cooperative detection method and system based on hierarchical knowledge reasoning, which is particularly suitable for omnidirectional visual detection of large-size and complex-structure industrial products such as new energy batteries. BACKGROUND

[0002] Knowledge reasoning refers to the process of deducing unknown knowledge based on existing knowledge through logical reasoning and induction. In a knowledge graph, knowledge reasoning specifically manifests as deducing new entity relationships or entity attributes using existing knowledge (such as triples) in the graph. With the development of artificial intelligence and big data technology, knowledge reasoning will be increasingly widely applied in various fields and become an indispensable part of artificial intelligence systems.

[0003] With the development of large-scale and integrated industrial products such as new energy batteries, single-camera detection schemes have been unable to meet the omnidirectional detection needs. Currently, scholars at home and abroad have conducted extensive research on multi-camera cooperative detection, which can improve the efficiency and intelligence level of industrial product detection.

[0004] However, existing multi-camera cooperative detection methods do not execute different control strategies for each detection level of the different detection processes of workpieces, making it difficult to adapt to different detection scenarios and needs. Therefore, there is an urgent need to develop an efficient and intelligent multi-camera cooperative detection method to adapt to different detection scenarios and needs and improve the detection performance of complex industrial products. SUMMARY

[0005] To address the issue that existing multi-camera cooperative detection methods do not execute different control strategies for each detection level of the different detection processes of workpieces, making it difficult to adapt to different detection scenarios and needs, the present application provides a multi-camera cooperative detection method and system based on hierarchical knowledge reasoning, which executes different control strategies for each detection level of the different detection processes of workpieces, automatically and efficiently acquires and processes a large amount of new knowledge data, and mines potential regularity features to adapt to different detection scenarios and needs. The specific technical solutions are as follows:

[0006] A multi-camera cooperative detection method based on hierarchical knowledge reasoning includes the following steps:

[0007] Relationship extraction is performed on each detection level data of the different detection processes of workpieces, triples data of each detection level data are acquired, and a hierarchical knowledge graph is constructed based on the acquired triples data of each detection level data.

[0008] Based on the inspection area where the workpiece is located, the position information of multiple cameras is obtained, and a multi-camera spatial relationship model is constructed based on the position information of multiple cameras;

[0009] Build a knowledge reasoning rule base and a dynamic knowledge update mechanism;

[0010] Extract key information from the artifact detection task, and based on this information, call the knowledge reasoning rule base and dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph;

[0011] In response to the workpiece detection instruction, the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning are used to control the actions of multiple cameras, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

[0012] The multi-camera collaborative detection method based on hierarchical knowledge reasoning constructs a hierarchical knowledge graph and a multi-camera spatial relationship model, calls a knowledge reasoning rule base and a dynamic knowledge update mechanism according to key information in the workpiece detection task, updates the hierarchical knowledge graph through knowledge reasoning, and controls the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain images of the workpiece to be detected. Based on hierarchical knowledge reasoning, it can execute different control strategies for each detection level of different detection processes of the workpiece, and automatically and efficiently acquire and process a large amount of new knowledge data, explore its potential regular characteristics, and adapt to different detection scenarios and needs, thereby realizing all-round visual inspection of large-size and complex structure industrial products such as new energy batteries, and improving the detection performance, efficiency and intelligence of complex industrial products.

[0013] Preferably, the specific method of constructing a knowledge reasoning rule base includes the following steps:

[0014] Get artifact properties;

[0015] Obtain testing requirements;

[0016] Obtain preset inference rules;

[0017] Build a knowledge reasoning rule library based on artifact attributes, detection requirements, and preset reasoning rules;

[0018] Among them, the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

[0019] Preferably, the specific method of constructing a dynamic knowledge update mechanism includes the following steps:

[0020] Obtain multi-dimensional feature values ​​of data at each detection level;

[0021] Calculate the update weight of each detection level based on the multi-dimensional feature values;

[0022] Set corresponding update thresholds for each detection level, each detection process, and the overall detection process;

[0023] Calculate the total weight of each detection process update and the total weight of the entire detection process based on the updated weight of each detection level;

[0024] According to the total weight of each detection process update, the total weight of the overall detection process and the corresponding update threshold, it is determined whether the knowledge update mechanism is triggered.

[0025] Preferably, the specific method of constructing a multi-camera spatial relationship model based on the position information of multiple cameras includes the following steps:

[0026] Camera point clustering is performed based on the location information and attribute values ​​of multiple cameras to divide different camera space areas;

[0027] For each inspection level of different inspection processes of the workpiece, several cameras in the camera space area are constrained to different degrees of spatial relationships according to preset constraints;

[0028] Based on digital modeling technology, a spatial relationship model of several cameras in different camera space areas is constructed.

[0029] Preferably, the knowledge update mechanism includes incremental update, real-time update, knowledge fusion, knowledge reasoning and knowledge evolution.

[0030] A multi-camera collaborative detection system based on hierarchical knowledge reasoning, including:

[0031] The first construction module is used to extract relationships from each detection layer data of different detection processes of the workpiece, obtain triple data of each detection layer data, and construct a hierarchical knowledge graph based on the obtained triple data of each detection layer data;

[0032] The second construction module is used to obtain the position information of multiple cameras based on the inspection area where the workpiece is located, and to construct a multi-camera spatial relationship model according to the position information of the multiple cameras;

[0033] The third building module is used to build a knowledge reasoning rule base and a dynamic knowledge update mechanism;

[0034] The knowledge dynamic update module is used to extract key information from the artifact detection task. Based on this key information, it calls the knowledge reasoning rule base and the dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph.

[0035] The detection module is used to respond to workpiece detection instructions, control the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

[0036] Preferably, the second building block comprises:

[0037] The area division unit is used to cluster camera points according to the location information and attribute values ​​of multiple cameras to divide different camera space areas;

[0038] The constraint unit is used to constrain the spatial relationship of several cameras in the camera space area to different degrees according to preset constraint conditions for each detection level of different detection processes of the workpiece;

[0039] The second construction unit is used to construct a spatial relationship model for a plurality of cameras in different camera space areas based on digital modeling technology.

[0040] Preferably, the third building block includes:

[0041] A first acquisition unit, configured to acquire workpiece attributes, detection requirements, and preset inference rules;

[0042] The third construction unit is used to construct a knowledge reasoning rule library based on artifact attributes, detection requirements, and preset reasoning rules;

[0043] Among them, the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

[0044] Preferably, the third building block further includes:

[0045] A second acquisition unit is used to obtain multi-dimensional feature values ​​of each detection level data;

[0046] A weight calculation unit is used to calculate the updated weight of each detection level according to the multi-dimensional feature value, and calculate the updated total weight of each detection process and the total weight of the entire detection process according to the updated weight of each detection level;

[0047] A threshold setting unit, used to set corresponding update thresholds for each detection level, each detection process, and the overall detection process;

[0048] The judgment unit is used to judge whether to trigger the knowledge update mechanism based on the total weight of each detection process update, the total weight of the entire detection process and the corresponding update threshold.

[0049] Preferably, the knowledge update mechanism includes incremental update, real-time update, knowledge fusion, knowledge reasoning and knowledge evolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but rather the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0051] Figure 1 This is a schematic diagram of the overall process of a multi-camera collaborative detection method based on hierarchical knowledge reasoning in one embodiment of the present invention;

[0052] Figure 2 1 is a flow chart of a specific method for constructing a multi-camera spatial relationship model in one embodiment of the present invention;

[0053] Figure 3 It is a flowchart of a specific method for constructing a knowledge reasoning rule base in one embodiment of the present invention;

[0054] Figure 4 It is a flowchart of a specific method for constructing a dynamic knowledge update mechanism in one embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the process of constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 1 ;

[0056] Figure 6 This is a schematic diagram of the process of constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 2 ;

[0057] Figure 7 This is a schematic diagram of the process of constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 3 ;

[0058] Figure 8 This is a schematic diagram of the process of constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 4 . DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with its embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0060] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly attached to the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only implementation methods.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0062] The "first" and "second" in the present invention do not represent specific quantities and orders, but are only used to distinguish names.

[0063] Before elaborating on the specific embodiments of the multi-camera collaborative detection method and system based on hierarchical knowledge reasoning described in the present invention, a brief introduction to the prior art is first given.

[0064] Regarding knowledge reasoning: Knowledge reasoning refers to the process of deriving unknown knowledge from existing knowledge through logical reasoning and inductive summarization. In knowledge graphs, knowledge reasoning specifically involves leveraging existing knowledge (such as triples) in the graph to derive new relationships between entities or entity attributes. With the development of artificial intelligence and big data technologies, the application of knowledge reasoning in various fields will become increasingly widespread, becoming an indispensable part of artificial intelligence systems.

[0065] Regarding knowledge graphs: Knowledge graphs, also known as knowledge domain visualization or knowledge domain mapping maps in the library and information science community, are a series of various graphs that display the development process and structural relationships of knowledge. They use visualization technology to describe knowledge resources and their carriers, and to mine, analyze, construct, map, and display knowledge and the connections between them. Specifically, knowledge graphs are a modern theory that achieves the goal of multidisciplinary integration by combining theories and methods from disciplines such as applied mathematics, graphics, information visualization technology, and information science with methods such as citation analysis and co-occurrence analysis in quantitative terms, and using visual graphs to vividly display the core structure, development history, cutting-edge fields, and overall knowledge architecture of a discipline. It displays complex knowledge domains through data mining, information processing, knowledge measurement, and graph drawing, revealing the dynamic development laws of knowledge domains and providing practical and valuable references for disciplinary research.

[0066] Regarding hierarchical knowledge reasoning: Hierarchical knowledge reasoning can be understood as a method that breaks down the knowledge reasoning process into multiple levels, each handling different tasks and goals, thereby gradually deriving the final conclusion. This reasoning method typically involves multiple steps and levels, each with its own specific tasks and goals. Through step-by-step reasoning and decision-making, the overall knowledge reasoning goal is ultimately achieved. Hierarchical knowledge reasoning can decompose complex tasks into multiple simple subtasks, facilitating modular and parallel processing. The reasoning results of each level can serve as input to the next level, allowing for gradual verification and correction, thereby improving the accuracy of the reasoning. In addition, the hierarchical structure makes the system easy to expand and maintain.

[0067] For large, complex industrial products like new energy batteries, single-camera inspection solutions cannot meet the comprehensive inspection requirements. Existing multi-camera collaborative inspection methods, which are large in size and complex in structure, are unable to automatically and efficiently acquire and process large amounts of new knowledge data and uncover its underlying patterns. Consequently, they struggle to implement different control strategies for each inspection layer of the workpiece's different inspection processes, making them incapable of adapting to diverse inspection scenarios and requirements.

[0068] In order to solve the above problems, Figure 1 As shown, the present invention provides a multi-camera collaborative detection method based on hierarchical knowledge reasoning, which includes the following steps:

[0069] S1, extract relations from each detection level data of different detection processes of the workpiece, obtain triple data of each detection level data, and construct a hierarchical knowledge graph based on the triple data of each detection level data obtained.

[0070] The different inspection processes of a workpiece can be different production stages of the workpiece, such as the assembly, quality inspection, packaging, and transportation of new energy batteries. The inspection level can be understood as the inspection objectives and tasks targeted by a specific inspection process, such as surface defect inspection (such as side and / or bottom), size and type inspection, yield inspection, component installation inspection, surface color and texture inspection, and physical and mechanical property inspection.

[0071] Each inspection layer's data can be obtained based on historical data collected during the workpiece inspection process. Relationships are extracted from each layer's data to obtain corresponding triples. A hierarchical knowledge graph is constructed based on these triples. This facilitates decomposing knowledge reasoning into multiple layers based on the hierarchical knowledge graph. This allows for step-by-step reasoning at each inspection layer, improving the accuracy of data reasoning at each inspection layer and ultimately achieving accuracy for each inspection process and overall inspection.

[0072] S2, based on the detection area where the workpiece is located, obtains the position information of multiple cameras and constructs a multi-camera spatial relationship model based on the position information of multiple cameras.

[0073] Specifically, a spatial coordinate system is constructed based on the inspection area where the workpiece is located, and the position information of the camera is obtained according to the spatial coordinate system and the specific position of the camera. The multi-camera spatial relationship model can be characterized as the mutual constraints of the spatial relationship of several cameras in different spatial areas. For example, according to the image acquisition purpose of several cameras, the relationship between the image acquisition areas of several cameras (including area size, area location and the amount of area overlap, etc.), the number of image acquisitions and the image acquisition angle are bound and limited to achieve full-face image acquisition of the entire workpiece; or according to the different inspection levels under each inspection process, the depth and / or width of the inspection level targeted by the image acquisition of several cameras are restricted by relationship constraints. For some inspection levels, a preset number of cameras are deployed for image acquisition, and for other inspection levels, all cameras in the spatial area are called to acquire image information. The depth of the inspection level mentioned here includes but is not limited to the image pixel depth and images of different sizes at multiple angles. The width of the inspection level includes but is not limited to the area size of the image, such as the local image and the overall image of a certain part of the workpiece. The spatial relationship of several cameras in different spatial areas is constrained in order to coordinate the cameras in the spatial area as a whole, so as to use the cameras in a targeted manner to conduct comprehensive overall inspection of different inspection levels of each inspection process of the workpiece, and to implement different control strategies for each inspection level of the different inspection processes of the workpiece to realize the multi-camera collaborative inspection function.

[0074] Preferably, in step S2, as Figure 2 As shown, the specific method of constructing a multi-camera spatial relationship model based on the position information of multiple cameras includes the following steps:

[0075] S20: performing camera point clustering according to the location information and attribute values ​​of the multiple cameras to divide different camera space areas.

[0076] Camera point clustering methods include, but are not limited to, density-based clustering and K-means partitioning. Here, cameras can be classified based on their attributes, such as maximum pixel count, age, design life, distance to the workpiece, and accuracy. Cluster centers and K values ​​are then preset based on the location information of multiple cameras. Finally, K-means clustering is used to cluster the camera points of multiple cameras to create distinct camera space regions. Alternatively, the radius distance r and minPoints can be determined first, and then density-based clustering can be used to cluster the camera points of multiple cameras to create distinct camera space regions.

[0077] By clustering multiple cameras and dividing them into different camera space areas, camera resources can be allocated with a focus as needed for different detection levels of different detection processes, so as to better control and manage multiple cameras and improve the efficiency of their collaborative detection.

[0078] S21 , for each detection level of different detection processes of the workpiece, performing spatial relationship constraints of different degrees on a plurality of cameras in the camera space area according to preset constraint conditions.

[0079] Preset conditions include, but are not limited to, constraints between the image acquisition areas, number of images acquired, and image acquisition angles of the cameras. Preferably, the spatial relationship constraints include constraints between the image acquisition areas, number of images acquired, and image acquisition angles of the cameras. Constraints between image acquisition areas include constraints on area size, area location, and area overlap. For a specific inspection level, the total area of ​​the acquisition areas of the cameras and the range of the image acquisition areas of the main cameras are first set. The remaining cameras act as auxiliary cameras to follow the main camera, and their acquisition areas vary based on the total area of ​​the acquisition areas and the size of the image acquisition areas of the main cameras. Furthermore, the overlap area range, minimum overlap area, or maximum overlap area of ​​the image acquisition areas of the cameras can also be agreed upon. By constraining the spatial relationships of the cameras in different spatial areas, the cameras within the spatial area can be comprehensively deployed, allowing targeted use of the cameras to conduct comprehensive, integrated inspections at different inspection levels of each workpiece inspection process. Different control strategies can be implemented for each inspection level of the workpiece inspection process, enabling multi-camera collaborative inspection and improving multi-camera collaborative work efficiency.

[0080] S22, based on digital modeling technology, constructs a spatial relationship model for several cameras in different camera space areas.

[0081] Based on digital modeling technology, a spatial relationship model is constructed to digitally control several cameras in different camera space areas to achieve collaborative detection.

[0082] S3, builds a knowledge reasoning rule base and a dynamic knowledge update mechanism.

[0083] Preferably, if Figure 3 As shown in FIG, the specific method of constructing a knowledge reasoning rule base includes the following steps:

[0084] S30, obtaining workpiece attributes. Workpiece attributes include, but are not limited to, type and size, mechanical properties, appearance and color, and batch. The inference rules for workpieces with different attributes vary. By obtaining workpiece attributes to build a knowledge inference rule library, it can be more practical.

[0085] S31: Obtain inspection requirements. Inspection requirements generally match workpiece attributes and include, but are not limited to, type and size detection, surface defect detection, and model and batch detection. Different inspection requirements correspond to different inference rules. For example, for workpiece type and size detection, forward inference rules are used to gradually match the similarity between workpiece images in the sample library and the collected workpiece images to determine the workpiece type and size. Of course, reverse inference rules can also be used to detect workpiece type and size.

[0086] S32, obtaining preset reasoning rules; wherein the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

[0087] S33, constructing a knowledge reasoning rule base based on the artifact attributes, detection requirements, and preset reasoning rules. Preferably, the knowledge reasoning rule base is constructed by constructing multiple mapping relationships between the artifact attributes, detection requirements, and preset reasoning rules to improve the richness and hierarchy of the knowledge reasoning rule base, specifically including the following:

[0088] 1. Such as Figure 5 As shown, according to the preset mapping relationship between workpiece attributes and detection requirements, the workpiece attributes are mapped one-to-one, one-to-many, or many-to-many to the detection requirements, and then according to the mapping relationship between the preset detection requirements and the preset inference rules, the detection requirements are mapped one-to-one, one-to-many, or many-to-many to the preset inference rules;

[0089] 2. Such as Figure 6 As shown, according to the mapping relationship between the preset detection requirements and the workpiece attributes, the detection requirements are mapped one-to-one, one-to-many or many-to-many to the workpiece attributes, and then according to the mapping relationship between the preset workpiece attributes and the preset reasoning rules, the workpiece attributes are mapped one-to-one, one-to-many or many-to-many to the preset reasoning rules;

[0090] 3. Such as Figure 7 As shown, according to the mapping relationship between preset artifact attributes and preset reasoning rules, the artifact attributes are mapped one-to-one, one-to-many or many-to-many to the preset reasoning rules;

[0091] 4. Such as Figure 8 As shown, according to the mapping relationship between the preset detection requirements and the preset reasoning rules, the detection requirements are mapped to the preset reasoning rules one-to-one, one-to-many or many-to-many.

[0092] Among them, A1-A10 represent different workpiece attributes, B1-B10 represent different detection requirements, and C1-C10 represent different preset inference rules.

[0093] In practical applications, different preset reasoning rules are used in view of different workpiece properties and detection requirements, which undoubtedly improves the efficiency and accuracy of reasoning and faster deduces new knowledge and conclusions. Therefore, by combining workpiece properties, detection requirements and preset reasoning rules to construct a knowledge reasoning rule library, the richness and hierarchy of the knowledge reasoning rule library can be improved, and the efficiency and accuracy of reasoning can be improved, and new knowledge and conclusions can be deduced faster.

[0094] S4, extracting key information in the workpiece detection task, according to the key information, calling the knowledge reasoning rule library and the dynamic knowledge updating mechanism, updating the hierarchical knowledge graph.

[0095] The key information can be a keyword in the workpiece detection task, such as a keyword corresponding to a specific workpiece property and / or detection requirement. By associating the preset key information with the workpiece property and / or detection requirement, the workpiece detection task is associated with the workpiece property and / or detection requirement.

[0096] When the key information is extracted, the dynamic knowledge updating mechanism is triggered, the preset reasoning rules in the knowledge reasoning rule library corresponding to the key information are called, the hierarchical knowledge graph is updated by knowledge reasoning, and new knowledge conclusions are obtained to expand the hierarchical knowledge graph.

[0097] S5, in response to the workpiece detection instruction, according to the multi-camera spatial relationship model and the updated hierarchical knowledge graph based on knowledge reasoning, controlling the action of the multiple cameras, obtaining the workpiece image to be detected, and based on the workpiece image to be detected, completing the detection task of the workpiece to be detected.

[0098] The workpiece detection instruction can be input from the outside, such as customer input, or actively triggered to generate, such as a preset workpiece detection instruction and a trigger condition, and real-time detection environment factors are obtained, when the detection environment factors meet the trigger condition, the workpiece detection instruction is automatically generated according to the multi-camera spatial relationship model and the updated hierarchical knowledge graph based on knowledge reasoning, the multiple cameras are controlled to work cooperatively to obtain the workpiece image to be detected, and based on the workpiece image to be detected, the detection task of the workpiece to be detected is completed.

[0099] According to the multi-camera spatial relationship model and the updated hierarchical knowledge graph through knowledge reasoning, the multiple cameras are controlled to work cooperatively, which can perform a detection task according to a detection requirement and / or a real-time dynamic change of a workpiece attribute. Since the detection task of the workpiece to be detected is completed through the cooperative work of the multiple cameras, and based on the hierarchical knowledge reasoning, different control strategies can be executed for each detection level of the different detection processes of the workpiece, a large amount of new knowledge data meeting the preset reasoning logic can be automatically and efficiently acquired and processed, and potential regular features thereof are mined to adapt to different detection scenes and requirements, so that flexible and all-around visual detection of large-size and complex-structure industrial products such as new energy batteries is realized.

[0100] The multi-camera cooperative detection method based on hierarchical knowledge reasoning can realize all-around visual detection of large-size and complex-structure industrial products such as new energy batteries by constructing a hierarchical knowledge graph and a multi-camera spatial relationship model, calling a knowledge reasoning rule library and a dynamic knowledge updating mechanism according to key information in a workpiece detection task, updating the hierarchical knowledge graph through knowledge reasoning, and controlling the actions of multiple cameras according to the multi-camera spatial relationship model and the updated hierarchical knowledge graph through knowledge reasoning, so that the detection performance, efficiency and intelligent degree of complex industrial products are improved.

[0101] As a preferred technical solution, in step S3, as shown in the figure, Figure 4 The specific method for constructing the dynamic knowledge updating mechanism includes the following steps:

[0102] S34, a multi-dimensional feature value of each detection level data is acquired. The multi-dimensional feature value refers to feature values of different types of data of the detection level, including but not limited to size type, surface defect pixel gray value, workpiece surface shape and curved surface area, defect position, part quantity and position, and color texture data. More specifically, for a new energy battery, the multi-dimensional feature value can be the appearance shape and texture color of the battery shell, the position and pixel gray value of defects such as scratches and pits, the size and shape of the battery cap, or the number of mounting hole positions on the outer surface, etc.

[0103] S35, the update weight of each detection level is calculated according to the multi-dimensional feature value. Specifically, each dimensional feature value is given a corresponding weight factor, and each dimensional weight value is obtained by calculating the product of the difference between each dimensional feature value and a preset standard dimensional feature value and the corresponding weight factor. Finally, the mean value of the dimensional weight values of multiple dimensions in the detection level is calculated to obtain the update weight.

[0104] S36: For each detection level, each detection process, and the entire detection process, a corresponding update threshold is set. The update threshold can be set by a technician based on experience.

[0105] S37, calculate the total updated weight of each detection process and the total weight of the overall detection process based on the updated weight of each detection level; specifically, the total updated weight of each detection process is equal to the weighted average of the updated weights of all detection levels under the detection process, and the total weight of the overall detection process is equal to the weighted average of the total updated weights of all detection processes.

[0106] S38: Based on the total update weight of each detection process, the total weight of the overall detection process, and the corresponding update threshold, a determination is made as to whether the knowledge update mechanism should be triggered. If the total update weight of one detection process is greater than the corresponding update threshold, or the total weight of the overall detection process is greater than the corresponding update threshold, indicating that a new knowledge conclusion has emerged, the knowledge update mechanism is triggered, and the knowledge inference rule base is invoked to perform knowledge inference updates on the hierarchical knowledge graph.

[0107] Furthermore, the key information in the workpiece detection task is extracted, and based on the key information, the knowledge reasoning rule library and the dynamic knowledge update mechanism are called to determine whether to perform knowledge reasoning updates on the hierarchical knowledge graph. At the same time, the total update weight of each detection process, the total weight of the overall detection process and the corresponding update threshold are calculated and compared at a preset frequency to determine whether the knowledge update mechanism is triggered. If so, the knowledge reasoning rule library is called to perform knowledge reasoning updates on the hierarchical knowledge graph.

[0108] Preferably, the knowledge update mechanism includes but is not limited to incremental update, real-time update, knowledge fusion, knowledge reasoning and knowledge evolution.

[0109] By building a dynamic knowledge update mechanism, timely knowledge reasoning and updating of the hierarchical knowledge graph can be performed to ensure the real-time and high efficiency of the multi-camera collaborative detection method.

[0110] The present invention also provides a multi-camera collaborative detection system based on hierarchical knowledge reasoning, including a first construction module, a second construction module, a third construction module, a knowledge dynamic update module and a detection module.

[0111] The first construction module is used to extract relationships between each detection level data of different detection processes of the workpiece, obtain triple data of each detection level data, and construct a hierarchical knowledge graph based on the triple data of each detection level data obtained; the second construction module is used to obtain the position information of multiple cameras based on the detection area where the workpiece is located, and construct a multi-camera spatial relationship model based on the position information of multiple cameras; the third construction module is used to construct a knowledge reasoning rule base and a dynamic knowledge update mechanism; the knowledge dynamic update module is used to extract key information in the workpiece detection task, and based on the key information, call the knowledge reasoning rule base and the dynamic knowledge update mechanism to perform knowledge reasoning and update the hierarchical knowledge graph; the detection module is used to respond to the workpiece detection instruction, control the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

[0112] Preferably, the second construction module includes a region division unit, a constraint unit and a second construction unit.

[0113] The area division unit is used to cluster camera points based on the position information and attribute values ​​of multiple cameras to divide different camera space areas; the constraint unit is used to impose different degrees of spatial relationship constraints on several cameras in the camera space area according to preset constraints for each detection level of different detection processes of the workpiece; the second construction unit is used to construct a spatial relationship model for several cameras in different camera space areas based on digital modeling technology.

[0114] The third building module includes a first acquiring unit and a third building unit.

[0115] The first acquisition unit is used to acquire workpiece attributes, detection requirements and preset reasoning rules; the third construction unit is used to construct a knowledge reasoning rule library according to the workpiece attributes, detection requirements and preset reasoning rules.

[0116] Workpiece attributes include, but are not limited to, type and size, mechanical properties, appearance and color, and batch. Inspection requirements are generally matched to workpiece attributes, including but not limited to type and size inspection, surface defect inspection, and model and batch inspection.

[0117] The preset reasoning rules include but are not limited to forward reasoning, backward reasoning and mixed reasoning.

[0118] The third building block further includes a second acquiring unit, a weight calculating unit, a threshold setting unit and a judging unit.

[0119] The second acquisition unit is used to obtain the multi-dimensional characteristic values ​​of each detection level data; the weight calculation unit is used to calculate the updated weight of each detection level based on the multi-dimensional characteristic values, and calculate the updated total weight of each detection process and the total weight of the entire detection process based on the updated weight of each detection level.

[0120] Specifically, the total update weight of each detection process is equal to the weighted average of the update weights of all detection levels under the detection process, and the total weight of the overall detection process is equal to the weighted average of the total update weights of all detection processes.

[0121] The threshold setting unit is used to set corresponding update thresholds for each detection level, each detection process and the overall detection process.

[0122] The judgment unit is used to determine whether to trigger the knowledge update mechanism based on the total update weight of each detection process, the total weight of the overall detection process, and the corresponding update threshold. If the total update weight of one detection process is greater than the corresponding update threshold or the total weight of the overall detection process is greater than the corresponding update threshold, indicating that a new knowledge conclusion has emerged, the knowledge update mechanism is triggered, and the knowledge inference rule base is called to perform knowledge inference updates on the hierarchical knowledge graph.

[0123] Knowledge updating mechanisms include but are not limited to incremental updating, real-time updating, knowledge fusion, knowledge reasoning, and knowledge evolution.

[0124] The multi-camera collaborative inspection system based on hierarchical knowledge reasoning constructs a hierarchical knowledge graph and a multi-camera spatial relationship model, calls a knowledge reasoning rule base and a dynamic knowledge update mechanism according to key information in the workpiece inspection task, updates the hierarchical knowledge graph through knowledge reasoning, and controls the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain images of the workpiece to be inspected. Based on hierarchical knowledge reasoning, it can execute different control strategies for each inspection level of different inspection processes of the workpiece, and automatically and efficiently acquire and process a large amount of new knowledge data, dig out its potential regular characteristics, and can adapt to different inspection scenarios and needs, realizing all-round visual inspection of large-size, complex-structure industrial products such as new energy batteries, thereby improving the inspection performance, efficiency and intelligence of complex industrial products.

[0125] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A multi-camera collaborative detection method based on hierarchical knowledge reasoning, characterized in that: The multi-camera collaborative detection method based on hierarchical knowledge reasoning includes the following steps: Relationship extraction is performed on each detection layer data of different detection processes of the workpiece to obtain triple data of each detection layer data, and based on the obtained triple data of each detection layer data, a hierarchical knowledge graph is constructed; Based on the inspection area where the workpiece is located, the position information of multiple cameras is obtained, and a multi-camera spatial relationship model is constructed based on the position information of multiple cameras; Build a knowledge reasoning rule base and a dynamic knowledge update mechanism; Extract key information from the artifact detection task, and based on this information, call the knowledge reasoning rule base and dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph; In response to the workpiece detection instruction, the system controls the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, acquires the image of the workpiece to be detected, and completes the inspection task of the workpiece to be detected based on the image of the workpiece to be detected; The specific method of building a knowledge reasoning rule base includes the following steps: Get artifact properties; Obtain testing requirements; Obtain preset inference rules; Build a knowledge reasoning rule library based on artifact attributes, detection requirements, and preset reasoning rules; The specific method of constructing a multi-camera spatial relationship model based on the position information of multiple cameras includes the following steps: Camera point clustering is performed based on the location information and attribute values ​​of multiple cameras to divide different camera space areas; For each inspection level of different inspection processes of the workpiece, several cameras in the camera space area are constrained to different degrees of spatial relationships according to preset constraints; Based on digital modeling technology, a spatial relationship model of several cameras in different camera space areas is constructed.

2. The multi-camera collaborative detection method based on hierarchical knowledge reasoning according to claim 1, characterized in that: The specific method of building a dynamic knowledge updating mechanism includes the following steps: Obtain multi-dimensional feature values ​​of data at each detection level; Calculate the update weight of each detection level based on the multi-dimensional feature values; Set corresponding update thresholds for each detection level, each detection process, and the overall detection process; Calculate the total weight of each detection process update and the total weight of the entire detection process based on the updated weight of each detection level; According to the total weight of each detection process update, the total weight of the overall detection process and the corresponding update threshold, it is determined whether the knowledge update mechanism is triggered.

3. The multi-camera collaborative detection method based on hierarchical knowledge reasoning according to claim 2 is characterized in that: Knowledge updating mechanisms include incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution.

4. A multi-camera collaborative detection system based on hierarchical knowledge reasoning, used to implement the multi-camera collaborative detection method according to any one of claims 1 to 3, characterized in that: The multi-camera collaborative detection system based on hierarchical knowledge reasoning includes: The first construction module is used to extract relationships from each detection layer data of different detection processes of the workpiece, obtain triple data of each detection layer data, and construct a hierarchical knowledge graph based on the obtained triple data of each detection layer data; The second construction module is used to obtain the position information of multiple cameras based on the inspection area where the workpiece is located, and to construct a multi-camera spatial relationship model according to the position information of the multiple cameras; The third building module is used to build a knowledge reasoning rule base and a dynamic knowledge update mechanism; The knowledge dynamic update module is used to extract key information from the artifact detection task. Based on this key information, it calls the knowledge reasoning rule base and the dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph. The detection module is used to respond to the workpiece detection instruction, control the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected; The second building block includes: The area division unit is used to cluster camera points according to the location information and attribute values ​​of multiple cameras to divide different camera space areas; The constraint unit is used to impose different degrees of spatial relationship constraints on several cameras in the camera space area according to preset constraint conditions for each detection level of different detection processes of the workpiece; A second construction unit is used to construct a spatial relationship model for a plurality of cameras in different camera space areas based on digital modeling technology; The third building block includes: A first acquisition unit, configured to acquire workpiece attributes, detection requirements, and preset inference rules; The third construction unit is used to construct a knowledge reasoning rule library based on artifact attributes, detection requirements and preset reasoning rules.

5. The multi-camera collaborative detection system based on hierarchical knowledge reasoning according to claim 4 is characterized in that: The preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

6. The multi-camera collaborative detection system based on hierarchical knowledge reasoning according to claim 5, characterized in that: The third building block also includes: A second acquisition unit is used to obtain multi-dimensional feature values ​​of each detection level data; A weight calculation unit is used to calculate the updated weight of each detection level according to the multi-dimensional feature value, and calculate the updated total weight of each detection process and the total weight of the entire detection process according to the updated weight of each detection level; A threshold setting unit, used to set corresponding update thresholds for each detection level, each detection process, and the overall detection process; The judgment unit is used to judge whether to trigger the knowledge update mechanism based on the total weight of each detection process update, the total weight of the entire detection process and the corresponding update threshold.

7. The multi-camera collaborative detection system based on hierarchical knowledge reasoning according to claim 6, characterized in that: Knowledge updating mechanisms include incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution.

Images