Multi-camera cooperative detection method and system based on hierarchical knowledge reasoning

Abstract

The invention relates to the technical field of intelligent manufacturing and machine vision, and provides a multi-camera cooperative detection method and system based on hierarchical knowledge reasoning, and the method comprises the steps: constructing a hierarchical knowledge graph; constructing a multi-camera space relation model; constructing a knowledge inference rule base and a dynamic knowledge updating mechanism; calling a knowledge reasoning rule base and a dynamic knowledge updating mechanism, and performing knowledge reasoning updating on the hierarchical knowledge graph; and controlling a plurality of cameras to act according to the multi-camera space relation model and the hierarchical knowledge graph after knowledge reasoning update, and completing a detection task of the to-be-detected workpiece. Different control strategies can be executed for each detection layer of different detection processes of the workpiece, a large amount of new knowledge data can be automatically and efficiently acquired and processed, potential law characteristics of the knowledge data can be mined, different detection scenes and requirements can be adapted, and the detection efficiency is improved. Omnibearing visual inspection of large-size and complex-structure industrial products such as new energy batteries is realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing and machine vision technology, and in particular to a multi-camera collaborative detection method and system based on hierarchical knowledge reasoning, which is particularly suitable for all-round visual inspection of large-size and complex-structure industrial products such as new energy batteries. Background Art

[0002] Knowledge reasoning refers to the process of deriving unknown knowledge through logical reasoning and induction based on existing knowledge. In the knowledge graph, knowledge reasoning is specifically manifested as using existing knowledge (such as triples) in the graph to derive new entity relationships or entity attributes. With the development of artificial intelligence and big data technology, the application of knowledge reasoning in various fields will become more and more extensive and become an indispensable part of artificial intelligence systems.

[0003] As industrial products such as new energy batteries develop towards large-scale and integrated development, single-camera detection solutions can no longer meet all-round detection needs. At present, domestic and foreign scholars have conducted extensive research on multi-camera collaborative detection, which can improve the efficiency and intelligence of industrial product detection.

[0004] However, the existing multi-camera collaborative inspection method does not implement different control strategies for each inspection level of the workpiece's different inspection processes, and it is difficult to adapt to different inspection scenarios and requirements. Therefore, it is urgent to develop an efficient and intelligent multi-camera collaborative inspection method to adapt to different inspection scenarios and requirements and improve the inspection performance of complex industrial products. Summary of the invention

[0005] Based on this, in order to solve the problem that the existing multi-camera collaborative detection method does not implement different control strategies for each detection level of different detection processes of the workpiece and is difficult to adapt to different detection scenarios and needs, the present invention provides a multi-camera collaborative detection method and system based on hierarchical knowledge reasoning, which implements different control strategies for each detection level of different detection processes of the workpiece, and automatically and efficiently acquires and processes a large amount of new knowledge data, and mines its potential regular features to adapt to different detection scenarios and needs. The specific technical solution is as follows:

[0006] A multi-camera collaborative detection method based on hierarchical knowledge reasoning includes the following steps:

[0007] Relationship extraction is performed on each detection level data of different detection processes of the workpiece to obtain triple data of each detection level data, and a hierarchical knowledge graph is constructed based on the triple data of each detection level data obtained;

[0008] Based on the inspection area where the workpiece is located, the position information of multiple cameras is obtained, and a multi-camera spatial relationship model is constructed according to the position information of the multiple cameras;

[0009] Build a knowledge reasoning rule base and a dynamic knowledge update mechanism;

[0010] Extract key information from the workpiece detection task, and based on the key information, call the knowledge reasoning rule base and dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph;

[0011] In response to the workpiece detection instruction, multiple camera actions are controlled according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain the image of the workpiece to be detected, and based on the image of the workpiece to be detected, the detection task of the workpiece to be detected is completed.

[0012] The multi-camera collaborative detection method based on hierarchical knowledge reasoning constructs a hierarchical knowledge graph and a multi-camera spatial relationship model, calls a knowledge reasoning rule base and a dynamic knowledge update mechanism according to key information in the workpiece detection task, updates the hierarchical knowledge graph through knowledge reasoning, and controls multiple camera actions according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain the image of the workpiece to be detected. Based on hierarchical knowledge reasoning, it can execute different control strategies for each detection level of different detection processes of the workpiece, and automatically and efficiently acquire and process a large amount of new knowledge data, dig out its potential regular characteristics, and can adapt to different detection scenarios and needs, realizing all-round visual inspection of large-size and complex structure industrial products such as new energy batteries, and improving the detection performance, efficiency and intelligence of complex industrial products.

[0013] Preferably, the specific method for constructing a knowledge reasoning rule base includes the following steps:

[0014] Get artifact properties;

[0015] Obtain testing requirements;

[0016] Obtain preset inference rules;

[0017] Build a knowledge reasoning rule library based on artifact attributes, detection requirements, and preset reasoning rules;

[0018] Among them, the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

[0019] Preferably, the specific method of constructing a dynamic knowledge updating mechanism includes the following steps:

[0020] Obtain multi-dimensional feature values ​​of each detection level data;

[0021] According to the multi-dimensional feature values, the update weight of each detection level is calculated;

[0022] Set corresponding update thresholds for each detection level, each detection process, and the overall detection process;

[0023] Calculate the total weight of each detection process update and the total weight of the overall detection process based on the update weight of each detection level;

[0024] According to the total update weight of each detection process, the total weight of the overall detection process and the corresponding update threshold, it is determined whether the knowledge update mechanism is triggered.

[0025] Preferably, the specific method of constructing a multi-camera spatial relationship model according to the position information of multiple cameras includes the following steps:

[0026] Camera point clustering is performed based on the location information and attribute values ​​of multiple cameras to divide different camera space areas;

[0027] For each inspection level of different inspection processes of the workpiece, several cameras in the camera space area are constrained to different degrees of spatial relationship according to preset constraints;

[0028] Based on digital modeling technology, a spatial relationship model of several cameras in different camera space areas is constructed.

[0029] Preferably, the knowledge updating mechanism includes incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution.

[0030] A multi-camera collaborative detection system based on hierarchical knowledge reasoning, comprising:

[0031] The first construction module is used to extract relations for each detection layer data of different detection processes of the workpiece, obtain triple data of each detection layer data, and construct a hierarchical knowledge graph based on the obtained triple data of each detection layer data;

[0032] The second construction module is used to obtain the position information of multiple cameras based on the inspection area where the workpiece is located, and to construct a multi-camera spatial relationship model according to the position information of the multiple cameras;

[0033] The third building module is used to build a knowledge reasoning rule base and a dynamic knowledge update mechanism;

[0034] The knowledge dynamic update module is used to extract key information in the workpiece detection task, and based on the key information, call the knowledge reasoning rule library and the dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph;

[0035] The detection module is used to respond to the workpiece detection instruction, control the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

[0036] Preferably, the second building block comprises:

[0037] A region division unit is used to perform camera point clustering according to the location information and attribute values ​​of multiple cameras to divide different camera space regions;

[0038] A constraint unit is used to constrain the spatial relationship of several cameras in the camera space area to different degrees according to preset constraint conditions for each detection level of different detection processes of the workpiece;

[0039] The second construction unit is used to construct a spatial relationship model for a plurality of cameras in different camera space areas based on a digital modeling technology.

[0040] Preferably, the third building block comprises:

[0041] A first acquisition unit, used to acquire workpiece attributes, detection requirements and preset reasoning rules;

[0042] The third construction unit is used to construct a knowledge reasoning rule library according to the workpiece attributes, detection requirements and preset reasoning rules;

[0043] Among them, the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

[0044] Preferably, the third building block further includes:

[0045] A second acquisition unit is used to acquire multi-dimensional feature values ​​of each detection level data;

[0046] A weight calculation unit is used to calculate the update weight of each detection level according to the multi-dimensional feature value, and calculate the total update weight of each detection process and the total weight of the overall detection process according to the update weight of each detection level;

[0047] A threshold setting unit, used to set corresponding update thresholds for each detection level, each detection process and the overall detection process;

[0048] The judgment unit is used to judge whether to trigger the knowledge update mechanism according to the total weight of each detection process update, the total weight of the overall detection process and the corresponding update threshold.

[0049] Preferably, the knowledge updating mechanism includes incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0051] Figure 1 It is a schematic diagram of the overall process of a multi-camera collaborative detection method based on hierarchical knowledge reasoning in one embodiment of the present invention;

[0052] Figure 2 is a flowchart of a specific method for constructing a multi-camera spatial relationship model in one embodiment of the present invention;

[0053] Figure 3 It is a flowchart of a specific method for constructing a knowledge reasoning rule base in one embodiment of the present invention;

[0054] Figure 4 It is a flowchart of a specific method for constructing a dynamic knowledge update mechanism in one embodiment of the present invention;

[0055] Figure 5 The present invention is a schematic diagram of a process for constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 1 ;

[0056] Figure 6 The present invention is a schematic diagram of a process for constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 2 ;

[0057] Figure 7 The present invention is a schematic diagram of a process for constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 3 ;

[0058] Figure 8 The present invention is a schematic diagram of a process for constructing a knowledge reasoning rule base by constructing multiple mapping relationships between workpiece attributes, detection requirements and preset reasoning rules in one embodiment of the present invention. Figure 4 . DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with its embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0060] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only and do not represent the only implementation method.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0062] The “first” and “second” mentioned in the present invention do not represent specific quantities and orders, but are merely used to distinguish names.

[0063] Before elaborating on the specific embodiments of the multi-camera collaborative detection method and system based on hierarchical knowledge reasoning described in the present invention, a brief introduction to the prior art is first given.

[0064] Regarding knowledge reasoning: Knowledge reasoning refers to the process of deriving unknown knowledge through logical reasoning and induction based on existing knowledge. In the knowledge graph, knowledge reasoning is specifically manifested as using existing knowledge in the graph (such as triples) to derive new entity relationships or entity attributes. With the development of artificial intelligence and big data technology, the application of knowledge reasoning in various fields will become more and more extensive and become an indispensable part of artificial intelligence systems.

[0065] Regarding the knowledge graph: The knowledge graph, known as knowledge domain visualization or knowledge domain mapping map in the library and information industry, is a series of various graphics that show the development process and structural relationship of knowledge. It uses visualization technology to describe knowledge resources and their carriers, and mines, analyzes, constructs, draws and displays knowledge and their mutual connections. Specifically, the knowledge graph is a modern theory that combines the theories and methods of applied mathematics, graphics, information visualization technology, information science and other disciplines with methods such as metrology citation analysis and co-occurrence analysis, and uses visualized graphs to vividly display the core structure, development history, frontier fields and overall knowledge architecture of the discipline to achieve the purpose of multidisciplinary integration. It displays complex knowledge fields through data mining, information processing, knowledge metrology and graphics drawing, reveals the dynamic development laws of knowledge fields, and provides practical and valuable references for disciplinary research.

[0066] Regarding hierarchical knowledge reasoning: hierarchical knowledge reasoning can be understood as a method of decomposing the knowledge reasoning process into multiple levels, each level handling different tasks and goals, and gradually deriving the final conclusion. This reasoning method usually involves multiple steps and levels, each level has its specific tasks and goals, and through step-by-step reasoning and decision-making, the overall knowledge reasoning goal is ultimately achieved. Hierarchical knowledge reasoning has the ability to decompose complex tasks into multiple simple subtasks, which is convenient for modularization and parallel processing. The reasoning results of each level can be used as the input of the next level, and can be gradually verified and corrected to improve the accuracy of reasoning. In addition, the hierarchical structure makes the system easy to expand and maintain.

[0067] For large-sized and complex-structured industrial products such as new energy batteries, single-camera inspection solutions cannot meet all-round inspection needs. Industrial products are large in size and complex in structure. The existing multi-camera collaborative inspection method cannot automatically and efficiently acquire and process a large amount of new knowledge data and mine its potential regular features. Therefore, it is difficult to implement different control strategies for each inspection level of the different inspection processes of the workpiece, and it cannot adapt to different inspection scenarios and needs.

[0068] In order to solve the above problems, Figure 1 As shown, the present invention provides a multi-camera collaborative detection method based on hierarchical knowledge reasoning, which includes the following steps:

[0069] S1, extract relations for each detection level data of different detection processes of the workpiece, obtain triple data of each detection level data, and construct a hierarchical knowledge graph based on the triple data of each detection level data obtained.

[0070] Different inspection processes of workpieces can be different production stages of workpieces, such as assembly, quality inspection, packaging and transportation of new energy batteries. The inspection level can be understood as the inspection target and task targeted in a certain inspection process, such as surface (such as side and / or bottom) defect inspection, size type inspection, yield inspection, component installation inspection, surface color and texture inspection, and physical and mechanical property inspection of workpieces.

[0071] Each detection level data can be obtained based on the historical data collected during the workpiece detection process. Relationships are extracted for each detection level data to obtain the corresponding triple data, and a hierarchical knowledge graph is constructed based on the triple data to facilitate the decomposition of knowledge reasoning into multiple different levels according to the hierarchical knowledge graph, and then step-by-step reasoning is performed for each detection level, improving the accuracy of data reasoning at each detection level, and ultimately achieving the accuracy of each detection process and even the overall detection.

[0072] S2, based on the inspection area where the workpiece is located, obtain the position information of multiple cameras, and build a multi-camera spatial relationship model according to the position information of the multiple cameras.

[0073] Specifically, a spatial coordinate system is constructed based on the inspection area where the workpiece is located, and the position information of the camera is obtained according to the spatial coordinate system and the specific position of the camera. The multi-camera spatial relationship model can be characterized as the mutual constraints of the spatial relationship of several cameras in different spatial areas. For example, according to the image acquisition purpose of several cameras, the relationship between the image acquisition areas of several cameras (including area size, area position and the amount of area overlap, etc.), the number of image acquisitions and the image acquisition angle are bound and limited to achieve the overall image acquisition of the workpiece; or according to the different detection levels under each detection process, the depth and / or breadth of the detection level targeted by the image acquisition of several cameras are restricted by relationship constraints. For some detection levels, a preset number of cameras are deployed for image acquisition, and for other detection levels, all cameras in the spatial area are called to acquire image information. The depth of the detection level mentioned here includes but is not limited to the image pixel depth and images of different sizes at multiple angles, and the breadth of the detection level includes but is not limited to the area size of the image, such as the local image and the overall image of a certain part of the workpiece. The spatial relationship of several cameras in different spatial areas is constrained in order to coordinate the cameras in the spatial area as a whole, so as to use the cameras to conduct comprehensive overall inspection of different inspection levels of each inspection process of the workpiece in a targeted manner, and to implement different control strategies for each inspection level of different inspection processes of the workpiece to realize multi-camera collaborative inspection function.

[0074] Preferably, in step S2, Figure 2 As shown, the specific method of constructing a multi-camera spatial relationship model based on the position information of multiple cameras includes the following steps:

[0075] S20, performing camera point clustering according to the location information and attribute values ​​of the multiple cameras to divide different camera space areas.

[0076] Camera point clustering methods include but are not limited to density-based clustering and K-means partitioning clustering. Here, the camera can be classified according to its attribute values, such as the maximum pixel, service life, design life, distance from the workpiece to be detected, accuracy, etc., and then the cluster center and K value are preset according to the location information of multiple cameras. Finally, the camera points of multiple cameras are clustered by the K-means clustering method to divide different camera space areas. The radius distance r and minPoints can also be determined first, and then the camera point distance of multiple cameras can be divided into different camera space areas by the density-based clustering method.

[0077] By clustering multiple cameras and dividing them into different camera space areas, camera resources can be allocated with emphasis as needed for different detection levels of different detection processes, so as to better control and manage multiple cameras and improve the efficiency of their collaborative detection.

[0078] S21, for each detection level of different detection processes of the workpiece, a plurality of cameras in the camera space area are constrained in different degrees of spatial relationship according to preset constraint conditions.

[0079] The preset conditions include, but are not limited to, constraints between the image acquisition areas, image acquisition numbers, and image acquisition angles of several cameras. Preferably, the spatial relationship constraints include constraints between the image acquisition areas, image acquisition numbers, and image acquisition angles of several cameras. The constraints between the image acquisition areas include constraints on area size, area position, and area overlap. For a certain detection level, the total area of ​​the acquisition areas of several cameras is set first, and the image acquisition area range of several main cameras is set. The remaining cameras are used as auxiliary cameras to follow the main camera for follow-up actions, and their acquisition areas change according to the total area of ​​the acquisition area and the size of the image acquisition areas of several main cameras. At the same time, the overlapping area range, minimum overlapping area, or maximum overlapping area of ​​the image acquisition areas of several cameras can also be agreed upon. In this way, by constraining the spatial relationship of several cameras in different spatial areas, the cameras in the spatial area can be deployed as a whole, so as to use the cameras in a targeted manner to perform a full range of overall detection of different detection levels of each detection process of the workpiece, and different control strategies are executed for each detection level of different detection processes of the workpiece, so as to realize the multi-camera collaborative detection function and improve the collaborative work efficiency of multiple cameras.

[0080] S22, based on digital modeling technology, construct a spatial relationship model for a plurality of cameras in different camera space areas.

[0081] Based on digital modeling technology, a spatial relationship model is constructed to digitally control several cameras in different camera space areas to achieve collaborative detection.

[0082] S3, builds a knowledge reasoning rule base and a dynamic knowledge updating mechanism.

[0083] Preferably, if Figure 3 As shown, the specific method of constructing a knowledge reasoning rule base includes the following steps:

[0084] S30, obtaining workpiece attributes; workpiece attributes include but are not limited to type and size, mechanical properties, appearance color and batch. For workpieces with different attributes, the reasoning rules will be different. By obtaining the workpiece attributes to build a knowledge reasoning rule library, it can be more practical.

[0085] S31, obtaining the inspection requirements; the inspection requirements are generally matched with the workpiece attributes, including but not limited to type and size inspection, surface defect inspection, and model batch inspection. Different inspection requirements have different corresponding reasoning rules. For example, for the type and size inspection of the workpiece, the forward reasoning rule is used to use the workpiece images in the sample library and the collected workpiece images to match the similarity one by one step by step, so that the type and size of the workpiece can be detected. Of course, the reverse reasoning rule can also be used to detect the type and size of the workpiece.

[0086] S32, obtaining preset reasoning rules; wherein the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

[0087] S33, constructing a knowledge reasoning rule base according to the artifact attributes, detection requirements and preset reasoning rules; preferably, constructing the knowledge reasoning rule base by constructing a multiple mapping relationship between the artifact attributes, detection requirements and preset reasoning rules to improve the richness and level of the knowledge reasoning rule base, specifically including the following:

[0088] 1. As Figure 5 As shown, according to the mapping relationship between the preset workpiece attributes and the detection requirements, the workpiece attributes are mapped one-to-one, one-to-many or many-to-many to the detection requirements, and then according to the mapping relationship between the preset detection requirements and the preset reasoning rules, the detection requirements are mapped one-to-one, one-to-many or many-to-many to the preset reasoning rules;

[0089] 2. If Figure 6 As shown, according to the mapping relationship between the preset detection requirements and the workpiece attributes, the detection requirements are mapped to the workpiece attributes one-to-one, one-to-many or many-to-many, and then according to the mapping relationship between the preset workpiece attributes and the preset reasoning rules, the workpiece attributes are mapped to the preset reasoning rules one-to-one, one-to-many or many-to-many;

[0090] 3. Such as Figure 7 As shown, according to the mapping relationship between preset artifact attributes and preset reasoning rules, the artifact attributes are mapped one-to-one, one-to-many, or many-to-many to the preset reasoning rules;

[0091] 4. Such as Figure 8 As shown, according to the mapping relationship between the preset detection requirements and the preset reasoning rules, the detection requirements are mapped to the preset reasoning rules one-to-one, one-to-many or many-to-many.

[0092] Among them, A1-A10 represent different workpiece attributes, B1-B10 represent different detection requirements, and C1-C10 represent different preset inference rules.

[0093] In practical applications, considering the different attributes of workpieces and detection requirements, the targeted use of different preset reasoning rules can undoubtedly improve the efficiency and accuracy of reasoning and derive new knowledge and conclusions more quickly. Therefore, by combining the attributes of workpieces, detection requirements and preset reasoning rules to build a knowledge reasoning rule base, it can not only improve the richness and level of the knowledge reasoning rule base, but also improve the efficiency and accuracy of reasoning, and derive new knowledge and conclusions more quickly.

[0094] S4, extracts the key information in the workpiece detection task, and based on the key information, calls the knowledge reasoning rule base and the dynamic knowledge update mechanism to perform knowledge reasoning and update the hierarchical knowledge graph.

[0095] The key information may be a keyword in the workpiece detection task, such as a keyword corresponding to a specific workpiece attribute and / or detection requirement. By associating the preset key information with the workpiece attribute and / or detection requirement, the workpiece detection task is associated with the workpiece attribute and / or detection requirement.

[0096] When the key information is extracted, the dynamic knowledge update mechanism is triggered, and the preset reasoning rules in the knowledge reasoning rule library corresponding to the key information are called to perform knowledge reasoning update on the hierarchical knowledge graph to obtain new knowledge conclusions and expand the hierarchical knowledge graph.

[0097] S5, in response to the workpiece detection instruction, controls the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, obtains the image of the workpiece to be detected, and completes the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

[0098] Workpiece detection instructions can be input externally, such as customer input, or actively triggered, such as pre-setting workpiece detection instructions and trigger conditions, and obtaining detection environment factors in real time. When the detection environment factors meet the trigger conditions, the workpiece detection instructions are automatically generated according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, and multiple cameras are controlled to work together to obtain the image of the workpiece to be detected, and based on the image of the workpiece to be detected, the detection task of the workpiece to be detected is completed.

[0099] According to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, multiple cameras are controlled to work together, which can perform inspection tasks according to the inspection requirements and / or the real-time dynamic changes of the workpiece attributes. Since the inspection task of the workpiece to be inspected is completed by the collaboration of multiple cameras, based on hierarchical knowledge reasoning, different control strategies can be implemented for each inspection level of the different inspection processes of the workpiece, and a large amount of new knowledge data that conforms to the preset reasoning logic can be automatically and efficiently acquired and processed, and its potential regular features can be excavated to adapt to different inspection scenarios and requirements, realizing flexible and all-round visual inspection of large-size and complex structure industrial products such as new energy batteries.

[0100] The multi-camera collaborative detection method based on hierarchical knowledge reasoning constructs a hierarchical knowledge graph and a multi-camera spatial relationship model, calls a knowledge reasoning rule base and a dynamic knowledge update mechanism according to key information in the workpiece detection task, updates the hierarchical knowledge graph through knowledge reasoning, and controls multiple camera actions according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain the image of the workpiece to be detected. Based on hierarchical knowledge reasoning, it can execute different control strategies for each detection level of different detection processes of the workpiece, and automatically and efficiently acquire and process a large amount of new knowledge data, dig out its potential regular characteristics, and can adapt to different detection scenarios and needs, realizing all-round visual inspection of large-size and complex structure industrial products such as new energy batteries, and improving the detection performance, efficiency and intelligence of complex industrial products.

[0101] As a preferred technical solution, in step S3, as Figure 4 As shown, the specific method of constructing a dynamic knowledge updating mechanism includes the following steps:

[0102] S34, obtaining multi-dimensional feature values ​​of each detection layer data. The multi-dimensional feature values ​​refer to the feature values ​​of different types of data at the detection layer, including but not limited to size type, surface defect pixel grayscale value, workpiece surface shape and curved surface area, defect location, component quantity and location, and color texture data. More specifically, for new energy batteries, the multi-dimensional feature values ​​can be the appearance shape and texture color of the battery shell, the location and pixel grayscale value of defects such as scratches and pits, the size and shape of the battery cap or the number of mounting holes on the outer surface, etc.

[0103] S35, calculate the updated weight of each detection level according to the multi-dimensional feature values; specifically, each dimensional feature value is assigned a corresponding weight factor, and the weight value of each dimension is obtained by calculating the difference between each dimensional feature value and the preset standard dimensional feature value and the product of the corresponding weight factor, and finally the average of the dimensional weight values ​​of multiple dimensions in the detection level is calculated to obtain the updated weight.

[0104] S36, setting a corresponding update threshold for each detection level, each detection process and the overall detection process. The update threshold can be set by a technician based on experience.

[0105] S37, calculate the total updated weight of each detection process and the total weight of the overall detection process according to the updated weight of each detection level; specifically, the total updated weight of each detection process is equal to the weighted average of the updated weights of all detection levels under the detection process, and the total weight of the overall detection process is equal to the weighted average of the total updated weights of all detection processes.

[0106] S38, judging whether to trigger the knowledge update mechanism according to the total update weight of each detection process, the total weight of the overall detection process and the corresponding update threshold. If the total update weight of one of the detection processes is greater than the corresponding update threshold or the total weight of the overall detection process is greater than the corresponding update threshold, indicating that a new knowledge conclusion has emerged, the knowledge update mechanism is judged to be triggered, and the knowledge reasoning rule base is called to perform knowledge reasoning update on the hierarchical knowledge graph.

[0107] Furthermore, the key information in the workpiece detection task is extracted, and based on the key information, the knowledge reasoning rule library and the dynamic knowledge update mechanism are called to determine whether to perform knowledge reasoning updates on the hierarchical knowledge graph. At the same time, the total update weight of each detection process, the total weight of the overall detection process and the corresponding update threshold are calculated and compared at a preset frequency to determine whether the knowledge update mechanism is triggered. If so, the knowledge reasoning rule library is called to perform knowledge reasoning updates on the hierarchical knowledge graph.

[0108] Preferably, the knowledge updating mechanism includes but is not limited to incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution.

[0109] By building a dynamic knowledge update mechanism, the hierarchical knowledge graph can be updated with timely knowledge reasoning to ensure the real-time and high efficiency of the multi-camera collaborative detection method.

[0110] The present invention also provides a multi-camera collaborative detection system based on hierarchical knowledge reasoning, including a first construction module, a second construction module, a third construction module, a knowledge dynamic update module and a detection module.

[0111] The first construction module is used to extract relations for each detection level data of different detection processes of the workpiece, obtain triple data of each detection level data, and construct a hierarchical knowledge graph based on the triple data of each detection level data obtained; the second construction module is used to obtain the position information of multiple cameras based on the detection area where the workpiece is located, and construct a multi-camera spatial relationship model according to the position information of multiple cameras; the third construction module is used to construct a knowledge reasoning rule base and a dynamic knowledge update mechanism; the knowledge dynamic update module is used to extract key information in the workpiece detection task, and based on the key information, call the knowledge reasoning rule base and the dynamic knowledge update mechanism to perform knowledge reasoning update on the hierarchical knowledge graph; the detection module is used to respond to the workpiece detection instruction, control the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated after knowledge reasoning, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

[0112] Preferably, the second construction module includes a region dividing unit, a constraint unit and a second construction unit.

[0113] The area division unit is used to cluster camera points according to the position information and attribute values ​​of multiple cameras to divide different camera space areas; the constraint unit is used to impose different degrees of spatial relationship constraints on several cameras in the camera space area according to preset constraints for each detection level of different detection processes of the workpiece; the second construction unit is used to construct a spatial relationship model for several cameras in different camera space areas based on digital modeling technology.

[0114] The third building module includes a first acquiring unit and a third building unit.

[0115] The first acquisition unit is used to acquire the workpiece attributes, detection requirements and preset reasoning rules; the third construction unit is used to construct a knowledge reasoning rule library according to the workpiece attributes, detection requirements and preset reasoning rules.

[0116] Workpiece attributes include, but are not limited to, type and size, mechanical properties, appearance and color, and batch. Inspection requirements are generally matched with workpiece attributes, including but not limited to type and size inspection, surface defect inspection, and model and batch inspection.

[0117] The preset reasoning rules include but are not limited to forward reasoning, backward reasoning and mixed reasoning.

[0118] The third building block also includes a second acquisition unit, a weight calculation unit, a threshold setting unit and a judgment unit.

[0119] The second acquisition unit is used to obtain the multi-dimensional characteristic values ​​of each detection level data; the weight calculation unit is used to calculate the update weight of each detection level according to the multi-dimensional characteristic values, and calculate the total update weight of each detection process and the total weight of the overall detection process according to the updated weight of each detection level.

[0120] Specifically, the total update weight of each detection process is equal to the weighted average of the update weights of all detection levels under the detection process, and the total weight of the overall detection process is equal to the weighted average of the total update weights of all detection processes.

[0121] The threshold setting unit is used to set the corresponding update threshold for each detection level, each detection process and the overall detection process.

[0122] The judgment unit is used to determine whether to trigger the knowledge update mechanism according to the total weight of each detection process update, the total weight of the overall detection process and the corresponding update threshold. If the total weight of one of the detection processes is greater than the corresponding update threshold or the total weight of the overall detection process is greater than the corresponding update threshold, indicating that a new knowledge conclusion has emerged, the knowledge update mechanism is judged to be triggered, the knowledge reasoning rule base is called, and the hierarchical knowledge graph is updated for knowledge reasoning.

[0123] Knowledge updating mechanisms include but are not limited to incremental updating, real-time updating, knowledge fusion, knowledge reasoning, and knowledge evolution.

[0124] The multi-camera collaborative inspection system based on hierarchical knowledge reasoning constructs a hierarchical knowledge graph and a multi-camera spatial relationship model, calls a knowledge reasoning rule base and a dynamic knowledge update mechanism according to key information in the workpiece inspection task, updates the hierarchical knowledge graph through knowledge reasoning, and controls the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain images of workpieces to be inspected. Based on hierarchical knowledge reasoning, it can execute different control strategies for each inspection level of different inspection processes of workpieces, and automatically and efficiently acquire and process a large amount of new knowledge data, dig out its potential regular features, and can adapt to different inspection scenarios and needs, realizing all-round visual inspection of large-size and complex-structure industrial products such as new energy batteries, and improving the inspection performance, efficiency and intelligence of complex industrial products.

[0125] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-camera collaborative detection method based on hierarchical knowledge reasoning, characterized in that: The multi-camera collaborative detection method based on hierarchical knowledge reasoning includes the following steps: Relationship extraction is performed on each detection level data of different detection processes of the workpiece to obtain triple data of each detection level data, and a hierarchical knowledge graph is constructed based on the triple data of each detection level data obtained; Based on the inspection area where the workpiece is located, the position information of multiple cameras is obtained, and a multi-camera spatial relationship model is constructed according to the position information of the multiple cameras; Build a knowledge reasoning rule base and a dynamic knowledge update mechanism; Extract key information from the workpiece detection task, and based on the key information, call the knowledge reasoning rule base and dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph; In response to the workpiece detection instruction, multiple camera actions are controlled according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning to obtain the image of the workpiece to be detected, and based on the image of the workpiece to be detected, the detection task of the workpiece to be detected is completed.

2. The multi-camera collaborative detection method based on hierarchical knowledge reasoning as claimed in claim 1, characterized in that: The specific method of building a knowledge reasoning rule base includes the following steps: Get artifact properties; Obtain testing requirements; Obtain preset inference rules; A knowledge reasoning rule library is constructed based on artifact attributes, detection requirements, and preset reasoning rules.

3. A multi-camera collaborative detection method based on hierarchical knowledge reasoning as claimed in claim 2, characterized in that: The specific method of constructing a dynamic knowledge updating mechanism includes the following steps: Obtain multi-dimensional feature values ​​of data at each detection level; According to the multi-dimensional feature values, the update weight of each detection level is calculated; Set corresponding update thresholds for each detection level, each detection process, and the overall detection process; Calculate the total weight of each detection process update and the total weight of the overall detection process based on the update weight of each detection level; According to the total update weight of each detection process, the total weight of the overall detection process and the corresponding update threshold, it is determined whether the knowledge update mechanism is triggered.

4. The multi-camera collaborative detection method based on hierarchical knowledge reasoning as claimed in claim 3, characterized in that: The specific method of constructing a multi-camera spatial relationship model based on the position information of multiple cameras includes the following steps: Camera point clustering is performed based on the location information and attribute values ​​of multiple cameras to divide different camera space areas; For each inspection level of different inspection processes of the workpiece, several cameras in the camera space area are constrained to different degrees of spatial relationship according to preset constraints; Based on digital modeling technology, a spatial relationship model of several cameras in different camera space areas is constructed.

5. The multi-camera collaborative detection method based on hierarchical knowledge reasoning as claimed in claim 4, characterized in that: Knowledge updating mechanisms include incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution.

6. A multi-camera collaborative detection system based on hierarchical knowledge reasoning, characterized in that: The multi-camera collaborative detection system based on hierarchical knowledge reasoning includes: The first construction module is used to extract relations for each detection layer data of different detection processes of the workpiece, obtain triple data of each detection layer data, and construct a hierarchical knowledge graph based on the obtained triple data of each detection layer data; The second construction module is used to obtain the position information of multiple cameras based on the inspection area where the workpiece is located, and to construct a multi-camera spatial relationship model according to the position information of the multiple cameras; The third building module is used to build a knowledge reasoning rule base and a dynamic knowledge update mechanism; The knowledge dynamic update module is used to extract key information in the workpiece detection task, and based on the key information, call the knowledge reasoning rule library and the dynamic knowledge update mechanism to perform knowledge reasoning and update on the hierarchical knowledge graph; The detection module is used to respond to the workpiece detection instruction, control the actions of multiple cameras according to the multi-camera spatial relationship model and the hierarchical knowledge graph updated by knowledge reasoning, obtain the image of the workpiece to be detected, and complete the detection task of the workpiece to be detected based on the image of the workpiece to be detected.

7. A multi-camera collaborative detection system based on hierarchical knowledge reasoning as claimed in claim 6, characterized in that: The second building block includes: A region division unit is used to perform camera point clustering according to the location information and attribute values ​​of multiple cameras to divide different camera space regions; A constraint unit is used to constrain the spatial relationship of several cameras in the camera space area to different degrees according to preset constraint conditions for each detection level of different detection processes of the workpiece; The second construction unit is used to construct a spatial relationship model for a plurality of cameras in different camera space areas based on a digital modeling technology.

8. The multi-camera collaborative detection system based on hierarchical knowledge reasoning as claimed in claim 7, characterized in that: The third building block includes: A first acquisition unit, used to acquire workpiece attributes, detection requirements and preset reasoning rules; The third construction unit is used to construct a knowledge reasoning rule library according to the workpiece attributes, detection requirements and preset reasoning rules; Among them, the preset reasoning rules include forward reasoning, backward reasoning and mixed reasoning.

9. A multi-camera collaborative detection system based on hierarchical knowledge reasoning as claimed in claim 8, characterized in that: The third building block also includes: A second acquisition unit is used to acquire multi-dimensional feature values ​​of each detection level data; A weight calculation unit is used to calculate the update weight of each detection level according to the multi-dimensional feature value, and calculate the total update weight of each detection process and the total weight of the overall detection process according to the update weight of each detection level; A threshold setting unit, used to set corresponding update thresholds for each detection level, each detection process and the overall detection process; The judgment unit is used to judge whether to trigger the knowledge update mechanism according to the total weight of each detection process update, the total weight of the overall detection process and the corresponding update threshold.

10. A multi-camera collaborative detection system based on hierarchical knowledge reasoning as claimed in claim 9, characterized in that: Knowledge updating mechanisms include incremental updating, real-time updating, knowledge fusion, knowledge reasoning and knowledge evolution.

Images