Fault detection method, device, electronic device and storage medium

Abstract

The present invention proposes a method, device, electronic device and storage medium for fault detection, which belongs to the technical field of electromechanical equipment. The method comprises: obtaining component information of a faulty component of an electromechanical device; obtaining a target image of the electromechanical device; the target image is an image obtained by shooting and containing the faulty component; marking the faulty component on the target image according to the component information; and displaying the marked target image. It should be understood that by marking and displaying the faulty component on the target image of the electromechanical device through the component information of the faulty component of the electromechanical device, the location of the faulty component can be intuitively displayed to professionals, making it convenient for non-professionals to locate the faulty component and troubleshoot the fault.

Description

Technical Field

[0001] The present invention belongs to the technical field of electromechanical equipment, and in particular relates to a fault detection method, device, electronic equipment and storage medium. Background Art

[0002] With the development of science and technology, more and more electromechanical equipment are used in various industries and even in home environments. These electromechanical equipment include but are not limited to robots, automated production equipment, communication equipment, printers, home appliances, automotive electronics, and fitness equipment.

[0003] Take robots as an example. It is inevitable that they will break down during use. Generally, professional technical maintenance personnel are required to go to the site for maintenance. Moreover, before the technical maintenance personnel complete the maintenance of the robot, the robot cannot work normally, which affects the robot's operating efficiency.

[0004] The current technology development trend is to improve the interchangeability of parts through standardization and modularization of parts, which will help reduce the cost of the entire industry. When a robot fails, it can be restarted by replacing parts of the same specification. In order to enable non-professionals to complete the replacement of modular parts, a convenient fault detection method is needed. Summary of the invention

[0005] The embodiments of the present invention provide a method, an apparatus, and an electronic device for fault detection, which can solve at least part of the above problems.

[0006] In a first aspect, an embodiment of the present invention provides a fault detection method, including:

[0007] Obtaining component information of a faulty component of an electromechanical device;

[0008] Acquire a target image of the electromechanical device; the target image is an image obtained by shooting and containing the faulty component;

[0009] marking the faulty component on the target image according to the component information;

[0010] The labeled target image is displayed.

[0011] Optionally, acquiring the target image of the electromechanical device includes:

[0012] Acquire a reference image of the area where the faulty component is located according to the component information;

[0013] acquiring a plurality of candidate images of the electromechanical device;

[0014] The candidate image with the highest matching degree with the reference image is used as the target image.

[0015] Optionally, acquiring the target image of the electromechanical device includes:

[0016] Acquire a reference image of the area where the faulty component is located according to the component information;

[0017] Acquire the captured candidate image, and if the matching degree between the candidate image and the reference image is greater than a preset threshold, use the candidate image as the target image;

[0018] If the degree of match between the candidate image and the reference image is less than or equal to the preset threshold, a prompt to reselect the shooting angle is issued, and an alternative image shot at an updated angle is obtained; until the degree of match between the candidate image and the reference image is greater than the preset threshold.

[0019] Optionally, the electromechanical device is provided with a plurality of camera devices;

[0020] Accordingly, obtaining a target image of the electromechanical device includes:

[0021] determining a target camera device among a plurality of camera devices according to the component information;

[0022] The target image is acquired through the target camera device, and the user is prompted with the location of the target camera device.

[0023] Optionally, the component information includes location information of the faulty component;

[0024] Accordingly, marking the faulty component on the target image according to the component information includes:

[0025] The faulty component is marked on the target image according to the position information.

[0026] Optionally, the position information includes relative position information between the faulty component and the characteristic structure;

[0027] Correspondingly, marking the faulty component on the target image according to the position information includes:

[0028] Identify the characteristic structure in the target image, and determine the basic coordinates of the target image according to the position of the characteristic structure in the target image;

[0029] Determine a first position of the faulty component in the target image according to the basic coordinates and the relative position information;

[0030] The faulty component is marked in the target image according to the first position.

[0031] Optionally, the component information further includes characteristic information of the faulty component;

[0032] Correspondingly, after determining the first position of the faulty component in the target image according to the basic coordinates and the relative position information, the method further includes:

[0033] Identify the target image according to the first position and the feature information, and determine a second position of the faulty component in the target image;

[0034] The first position is adjusted according to the second position information.

[0035] Optionally, the electromechanical device is a robot;

[0036] The marking the faulty component on the target image according to the component information includes:

[0037] The outline of the faulty component is marked on the target image according to the component information.

[0038] In a second aspect, an embodiment of the present invention provides a fault detection device, including:

[0039] A component information acquisition module, used to acquire component information of a faulty component of an electromechanical device;

[0040] A target image acquisition module, used to acquire a target image of the electromechanical device; the target image is an image obtained by shooting and containing the faulty component;

[0041] A target image marking module, used for marking the faulty component on the target image according to the component information;

[0042] The target image display module is used to display the marked target image.

[0043] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0044] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method steps described in the first aspect are implemented.

[0045] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, comprising: the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in the first aspect are implemented.

[0046] In a fifth aspect, an embodiment of the present invention provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method steps described in the first aspect above.

[0047] The fault detection method provided in the first aspect above can mark and display the faulty components on the target image of the electromechanical equipment through the component information of the faulty components of the electromechanical equipment, so as to intuitively show the location of the faulty components to professionals, making it convenient for non-professionals to locate the faulty components and troubleshoot the faults.

[0048] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 is a flowchart of a method for fault detection provided by an embodiment of the present invention;

[0051] Figure 2 is a flowchart of a fault detection method provided by another embodiment of the present invention;

[0052] Figure 3 is a flowchart of a fault detection method provided by another embodiment of the present invention;

[0053] Figure 4 is a flowchart of a fault detection method provided by another embodiment of the present invention;

[0054] Figure 5 is a structural schematic diagram of a fault detection device provided by an embodiment of the present invention;

[0055] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0057] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0058] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0059] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0060] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0061] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0062] Before describing the fault detection method provided by the embodiment of the present invention, to facilitate understanding of the embodiment of the present invention, the principle and inventive concept of the fault detection method provided by the embodiment of the present invention are described below.

[0063] With the development of science and technology, more and more electromechanical equipment are used in various industries and even in home environments. These electromechanical equipment include but are not limited to robots, automated production equipment, communication equipment, printers, home appliances, automotive electronics, and fitness equipment.

[0064] Take robots as an example. More and more industries are beginning to use robots to complete some of the work in their industries. Robots are bound to malfunction during use. Generally, professional technical maintenance personnel are required to go to the site for maintenance. Moreover, before the technical maintenance personnel complete the maintenance of the robot, the robot cannot work normally, which affects the robot's operating efficiency.

[0065] However, as robots are increasingly used, most scenarios require non-professionals who use robots to replace faulty robot parts. Due to the low technical literacy of non-professionals, they are unable to locate faulty parts in time, and thus are unable to replace faulty parts in time. Therefore, a fault detection method is needed that can provide non-professionals with guidance on how to replace parts while detecting faulty parts.

[0066] With the development of robot technology, the following three factors can be utilized to implement the fault detection method provided in the embodiment of the present invention to solve at least part of the above problems.

[0067] On the one hand, the current technological development trend is to improve the interchangeability of parts through standardization and modularization of parts. The standardization and modularization of parts also help to reduce the cost of the entire industry. In addition, standardized and modularized parts are also easy to replace in a timely manner. When the robot is abnormal, it can be restarted by replacing parts of the same specification.

[0068] Typically, standardized and modular components are connected to other components of the robot through pluggable interfaces or jacks. Standardized and modular components include but are not limited to resistors, capacitors, inductors, chips, and functional modules that implement a certain function, such as infrared ranging modules, audio sensor modules, etc.

[0069] Standardization and modularization are connected to other modules through standard interfaces, sockets and jacks. For example, the infrared distance detection component is connected to the robot's main controller motherboard through a pluggable interface. Another example is that the fuse is connected to the robot's power control board through a pluggable socket. Another example is that the replaceable resistor and capacitor components are connected to the main controller motherboard through a jack.

[0070] Secondly, by analyzing the power-on status of the robot's components and the results of the transmission data, the working status of the component can be judged, and then whether the component has failed. The basic principles include: when a component is damaged, the current value, voltage value, current value and voltage value changes will be different from the normal working state. By collecting the above parameters of each component and comparing them with the set detection threshold, the fault status of the component can be judged. In addition, some components, such as sensors, will feedback data to the robot's control chip. When the sensor is abnormal, the content, frequency and other parameters of the data it feedback will change. By detecting the changes in these sensor feedback data, the fault status of the component can also be judged. If a component is detected to be faulty, the relevant information of the component, such as location, type, model, appearance and other information, can be determined by querying the database.

[0071] Thirdly, in the reliability design of robots, some important components, such as data storage units, are usually designed with redundancy, so that even if the component fails, the robot can still complete all or part of its functions. For some minor components, such as distance sensors, when these components usually fail, the robot can still perform some functions, such as detection and communication of its own components.

[0072] Based on the above three factors, an embodiment of the present invention provides a method for fault detection, by obtaining component information of a faulty component of a robot; obtaining a target image of the robot; the target image is an image obtained by shooting and containing the faulty component; marking the faulty component on the target image according to the component information; and displaying the marked target image. While realizing the detection of component failure, a guide for convenient fault location is provided. When a robot fails, based on the above third factor, the robot can usually still realize some functions, so each of its components can be detected by the robot. Based on the above second factor, the robot can obtain component information of the faulty component by detecting the component interface, such as the location information of the faulty component, the characteristic information of the faulty component, the appearance information of the faulty component, the model information of the faulty component, the connection method information of the faulty component and other components, and the operation prompt information of the faulty component. On this basis, a target image containing the faulty component is obtained, and the faulty component in the image is marked by the above component information, so that the location of the faulty component can be intuitively shown to professionals. Based on the above first factor, non-professionals can locate the faulty component according to the marked faulty component image. The fault is eliminated by removing the faulty component and inserting a replacement component.

[0073] It should be pointed out that, in order to facilitate the understanding of the present invention, the above introduction to the inventive concept is based on a robot as an example. It should be understood that the objects or execution subjects of the fault detection methods provided in various embodiments of the present invention are applicable to various electromechanical equipment that meet the above three factors. The electromechanical equipment includes but is not limited to industrial electromechanical equipment, information electromechanical equipment, and people's livelihood electromechanical equipment, such as robots, automated production equipment, communication equipment, printers, household appliances, automotive electronic products, and fitness and sports machinery.

[0074] Based on the above inventive concept, an embodiment of the present invention provides a fault detection method. Figure 1 FIG. 1 is a flow chart of a method for fault detection provided by an embodiment of the present invention. The method can be applied to electromechanical equipment or terminal equipment and implemented by software and / or hardware of the electromechanical equipment or terminal equipment, such as Figure 1 As shown, the method includes steps S110 to S140. The specific implementation principle of each step is as follows:

[0075] S110, obtaining component information of a faulty component of the electromechanical equipment.

[0076] In some embodiments, the electromechanical device may be a robot. In other embodiments, the electromechanical device may include but is not limited to industrial electromechanical equipment, information electromechanical equipment, and livelihood electromechanical equipment, such as automated production equipment, communication equipment, printers, household appliances, automotive electronics, and fitness equipment.

[0077] The component information includes, but is not limited to, the location information of the faulty component, the characteristic information of the faulty component, the appearance information of the faulty component, the model information of the faulty component, the connection method information between the faulty component and other components, and the operation prompt information of the faulty component.

[0078] In an embodiment of the present invention, the electromechanical device detects a faulty component through the second factor in the discussion of the above inventive concept.

[0079] In one implementation of the present application, after the electromechanical equipment detects a faulty component, the component information of the faulty component is obtained by querying a data table or a database according to the type and number of the detected faulty component.

[0080] In another implementation of the present application, after receiving a notification that the electromechanical device has detected a faulty component, the terminal device obtains basic component information including but not limited to the type of electromechanical device, the type of faulty component, and the number of the faulty component detected by the electromechanical device from the electromechanical device by wireless or wired means. In a non-limiting manner, the terminal device can query the component information corresponding to the basic component information from its own storage medium or remote storage settings. The terminal device can also query the electromechanical device for the component information corresponding to the basic component information.

[0081] The fault detection method provided in the embodiment of the present invention can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present invention does not impose any restrictions on the specific type of terminal devices.

[0082] S120, acquiring a target image of the electromechanical equipment; the target image is an image obtained by shooting and containing a faulty component.

[0083] In an embodiment of the present invention, the electromechanical device or terminal device obtains a target image of the electromechanical device through a camera device. In order to make the target image an image obtained by shooting and containing a faulty component, the embodiment of the present invention provides the following three implementation methods.

[0084] In the first embodiment, the terminal device obtains the target image of the electromechanical device through a built-in or external camera device, which can be achieved in the following ways: according to the component information, obtain a reference image of the area where the faulty component is located; obtain multiple candidate images of the electromechanical device; and use the candidate image with the highest matching degree with the reference image as the target image. The area where the component is located can be a partial structural area of ​​the electromechanical device, or a circuit board where the faulty component is located. The reference image can be an image such as a contour map, a structure map, or a feature map of the faulty component that can be used to match the target image. The matching method can be any feature matching method based on image recognition. In a non-limiting manner, according to the component information, obtain a reference image of the area where the faulty component is located, which can be obtained from a pre-stored gallery based on the type and model information of the component information.

[0085] In the second embodiment, the terminal device obtains the target image of the electromechanical device through a built-in or external camera device, which can be achieved in the following ways: according to the component information, obtain a reference image of the area where the faulty component is located; obtain the captured alternative image, if the matching degree between the alternative image and the reference image is greater than a preset threshold, the alternative image is used as the target image; if the matching degree between the alternative image and the reference image is less than or equal to the preset threshold, a prompt to reselect the shooting angle is issued, and an alternative image captured at an updated angle is obtained; until the matching degree between the alternative image and the reference image is greater than the preset threshold. In a non-limiting specific example, a user uses a terminal device to take an alternative image, and the terminal device matches the reference with the alternative image, and obtains a matching degree of the alternative image that is lower than a preset threshold, for example, the matching degree is 40%, which is lower than the threshold of 80%. The terminal device issues a prompt message, prompting the user to change the angle for shooting. After the user changes the shooting angle for shooting, the above matching process is repeated until a target image matching the reference image is obtained.

[0086] It should be noted that the reference image can be stored in the storage medium of a terminal device, an electromechanical device, a remote server, etc., and a device that needs to use the reference image can obtain the reference image by communicating with the device storing the reference image.

[0087] In a third embodiment, the electromechanical equipment is provided with a plurality of cameras, and obtaining a target image of the electromechanical equipment can be achieved in the following manner: the electromechanical equipment determines a target camera among a plurality of cameras according to component information; obtains a target image through the target camera, and prompts a user of the location of the target camera. In a non-limiting manner, the electromechanical equipment can prompt a user of a target image of an area where a faulty component of the electromechanical equipment is located through a terminal device of the user, and obtain the target image taken by the terminal device through a wired or wireless communication method. Among them, prompting the user of the location of the target camera can be by displaying the location of the target camera through a display device. Alternatively, the location of the target camera can be prompted to the user by driving an indicator light of the target camera. In a non-limiting manner, the electromechanical equipment determines a target camera among a plurality of cameras according to component information, based on a preset correspondence between component information and cameras.

[0088] S130, marking the faulty component on the target image according to the component information.

[0089] The component information includes, but is not limited to, the location information of the faulty component, the characteristic information of the faulty component, the appearance information of the faulty component, the model information of the faulty component, the connection method information between the faulty component and other components, and the operation prompt information of the faulty component.

[0090] The annotation includes but is not limited to rectangular box annotation, prompt text annotation, outline annotation, and position prompt annotation. The rectangular box annotation is to mark the position of the faulty component in the target image through a rectangular box that can include the faulty component. The prompt text annotation is to indicate the position of the faulty component in the target image through text information. The position prompt annotation is to indicate the position of the faulty component in the target image through images such as arrows or pushpins.

[0091] In one implementation of the present invention, the component information includes location information of the faulty component. The electromechanical device or terminal device marks the faulty component on the target image according to the component information, which may be marking the faulty component on the target image according to the location information.

[0092] In another implementation of the embodiment of the present invention, the component information includes appearance information of the faulty component. The electromechanical device or terminal device marks the faulty component on the target image according to the component information, and the electromechanical device or terminal device can identify and search the target image, obtain a position that matches the appearance information of the faulty component, and mark the position of the target image.

[0093] In another implementation of the embodiment of the present invention, the component information includes model information of the faulty component. The electromechanical device or terminal device marks the faulty component on the target image according to the component information, and the electromechanical device or terminal device queries a database to obtain characteristic information of the faulty component according to the model information, and the electromechanical device or terminal device identifies and searches the target image, obtains a position that matches the characteristic information of the faulty component, and marks the position of the target image.

[0094] It should be noted that the method for image recognition or search in various implementations of the embodiments of the present invention may be to recognize and search for the target image through any recognition or search model.

[0095] In one implementation of this embodiment, marking the faulty component on the target image according to the component information includes: marking the outline of the faulty component on the target image according to the component information. It can be understood that by marking the outline of the faulty component on the target image, non-professionals can easily know the shape of the faulty component, thereby avoiding the occurrence of misoperation.

[0096] S140, displaying the labeled target image.

[0097] In the embodiment of the present invention, displaying the marked target image can be achieved in two ways.

[0098] In the first embodiment, the electromechanical device itself has a display device, and the marked target image can be displayed by the display device of the electromechanical device. In a non-limiting manner, the electromechanical device can also call the display device of the terminal device or other display devices to display the marked target image in a wired or wireless manner.

[0099] In a second embodiment, the marked target image may be displayed by a display device of the terminal device.

[0100] It can be understood that by marking and displaying the faulty component on the target image of the electromechanical equipment through its component information, the location of the faulty component can be intuitively displayed to professionals, making it easier for non-professionals to locate the faulty component and troubleshoot the fault.

[0101] exist Figure 1 Based on the embodiment of the present invention shown, Figure 2 Another embodiment of a fault detection method provided by an embodiment of the present invention is shown. Figure 2 As shown, in this embodiment, the component information includes location information of the faulty component. Figure 1 The step S130 of the fault detection method shown may be replaced by step S130 ′. Figure 2 The embodiment shown and Figure 1 The same steps included in the illustrated embodiment will not be described in detail here, please refer to Figure 1 The corresponding description of the illustrated embodiment.

[0102] S130', marking the faulty component on the target image according to the position information.

[0103] The position information may be absolute position information of the faulty component in the electromechanical device or relative position information of the faulty component with respect to other components of the electromechanical device.

[0104] In a non-limiting example of this embodiment, the position information includes relative position information between the faulty component and the characteristic structure. Non-limitingly, the position information is the relative position information between the interface of the faulty component and the characteristic structure. Then, if Figure 3 As shown, step S130', marking the faulty component on the target image according to the position information includes steps S131, S132 and S135.

[0105] S131, identifying a characteristic structure in a target image, and determining a basic coordinate of the target image according to a position of the characteristic structure in the target image.

[0106] The characteristic structure may be a device where the faulty component is located, such as an edge of a main controller circuit board. The characteristic structure may also be a pre-set structure for calibrating a position.

[0107] The electromechanical device or the terminal device determines the basic coordinates of the target image according to the position of the characteristic structure in the target image, and may use the position of the characteristic structure or the position of the geometric center of the characteristic structure as the basic coordinates.

[0108] S132: Determine a first position of the faulty component in the target image according to the basic coordinates and the relative position information.

[0109] The relative position information may be the position information formed by the distance and angle between the faulty component and the basic coordinates. The position information formed by the distance and angle between the faulty component of the electromechanical device or terminal device and the basic coordinates determines the first position of the faulty component in the target image.

[0110] S135: Mark the faulty component in the target image according to the first position.

[0111] In another non-limiting example of this embodiment, the component information further includes characteristic information of the faulty component. Figure 3 Based on the implementation method shown, Figure 4 As shown, after the first position of the faulty component in the target image is determined according to the basic coordinates and the relative position information, steps S133 and S134 are also included.

[0112] S133, identifying the target image according to the first position and the feature information, and determining a second position of the faulty component in the target image.

[0113] The feature information may be image features extracted by any feature extraction model. The features of each component may be extracted in advance and stored in electromechanical equipment or other storage media. The feature information matches the type, model and other information of the component. When it is necessary to obtain the feature information of the faulty component, the image features of the faulty component may be obtained from the storage medium through the type, model and other information of the detected faulty component. Optionally, the feature information may also be the geometric features of the component.

[0114] S134, adjusting the first position according to the second position information.

[0115] In the embodiments of the present invention, in some embodiments, it is necessary to mark the contour or geometric center of the faulty component. In these embodiments, the position information is the relative position information of the interface and the characteristic structure of the faulty component. The accurate geometric center or contour information of the component may not be obtained through the above-mentioned position information of the component information. After obtaining the approximate position of the faulty component through the position information and other means, the accurate position of the faulty component in the target image, that is, the second position information, can be obtained by identifying the faulty component. Adjusting the first position according to the second position information can be to update the first position information using the second position information.

[0116] It should be understood that the various implementations in the above embodiments can be logically combined. The order of the sequence numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0117] Corresponding to the above Figure 1 The fault detection method shown, Figure 5 The figure shows a fault detection device M100 provided by an embodiment of the present invention, comprising:

[0118] The component information acquisition module M110 is used to acquire component information of a faulty component of the electromechanical equipment.

[0119] The target image acquisition module M120 is used to acquire a target image of the electromechanical device; the target image is an image obtained by shooting and containing a faulty component.

[0120] The target image marking module M130 is used to mark the faulty component on the target image according to the component information.

[0121] The target image display module M140 is used to display the marked target image.

[0122] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-mentioned fault detection device M100 can be referred to Figures 1 to 4 The corresponding process of the method will not be repeated here.

[0123] It can be understood that the various implementation modes and implementation combinations in the above embodiments and their beneficial effects are also applicable to this embodiment and will not be described in detail here.

[0124] Figure 6 The structure diagram of an electronic device provided by an embodiment of the present invention is shown in FIG. The electronic device is used to implement the above-mentioned electromechanical device or terminal device. In some embodiments, the electromechanical device may be a robot. Figure 6 As shown, the electronic device D10 of this embodiment includes: at least one processor D100 ( Figure 6Only one is shown in the figure) a processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on at least one processor D100, and when the processor D100 executes the computer program D102, the steps in any of the above-mentioned method embodiments are implemented. Alternatively, when the processor D100 executes the computer program D102, the functions of each module / unit in the above-mentioned device embodiments are implemented. In some embodiments, the electronic device D10 further includes a camera device D103, a display device D104, and a communication device D105. The memory D101, the camera device D103, the display device D104, and the communication device D105 transmit information to and from the processor D100 via the bus D110.

[0125] The electronic device D10 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device D10 may also be an electromechanical device. In some embodiments, the electromechanical device may be a robot. The electronic device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art will appreciate that Figure 6 This is only an example of the electronic device D10 and does not constitute a limitation on the electronic device D10. The electronic device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0126] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0127] In some embodiments, the memory D101 may be an internal storage unit of the electronic device D10, such as a hard disk or memory of the electronic device D10. In other embodiments, the memory D101 may also be an external storage device of the electronic device D10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device D10. Further, the memory D101 may also include both an internal storage unit of the electronic device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of computer programs, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0128] The camera device D103 may be a camera built into the electronic device D10 or may be a camera external to the electronic device D10.

[0129] The display device D104 can be used to display various images, selection menus, and prompt information of information input by the user or information provided to the user. The display device D104 may include a display panel, and optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. The touch panel may cover the display panel, and when the touch panel detects a touch operation on or near it, it is transmitted to the processor D100 to determine the type of touch event, and then the processor D100 provides a corresponding visual output on the display panel according to the type of touch event.

[0130] The communication device D105 includes, but is not limited to, a communication module for implementing wireless communications such as cellular mobile communication, WiFi communication, and Bluetooth communication, and may also be a communication module for implementing wired communication.

[0131] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0132] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0133] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0134] An embodiment of the present invention provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0136] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] In the embodiments provided by the present invention, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for fault detection, characterized in that: include: Acquire component information of a faulty component of an electromechanical device, wherein the component information includes location information of the faulty component and feature information of the faulty component, and the electromechanical device is a robot; Acquire a target image of the electromechanical device; the target image is an image obtained by shooting and containing the faulty component; Marking the faulty component on the target image according to the component information includes: marking the faulty component on the target image according to the position information; Displaying the labeled target image; The position information includes relative position information between the faulty component and the characteristic structure, and marking the faulty component on the target image according to the position information includes: Identify the characteristic structure in the target image, and determine the basic coordinates of the target image according to the position of the characteristic structure in the target image; Determine a first position of the faulty component in the target image according to the basic coordinates and the relative position information; Identify the target image according to the first position and the feature information, and determine a second position of the faulty component in the target image; adjusting the first position according to the second position information; The faulty component is marked in the target image according to the first position.

2. The method according to claim 1, characterized in that The step of acquiring a target image of the electromechanical device comprises: Acquire a reference image of the area where the faulty component is located according to the component information; acquiring a plurality of candidate images of the electromechanical device; The candidate image with the highest matching degree with the reference image is used as the target image.

3. The method according to claim 1, characterized in that The step of acquiring a target image of the electromechanical device comprises: Acquire a reference image of the area where the faulty component is located according to the component information; Acquire the captured candidate image, and if the matching degree between the candidate image and the reference image is greater than a preset threshold, use the candidate image as the target image; If the degree of match between the candidate image and the reference image is less than or equal to the preset threshold, a prompt to reselect the shooting angle is issued, and an alternative image shot at an updated angle is obtained; until the degree of match between the candidate image and the reference image is greater than the preset threshold.

4. The method according to claim 1, characterized in that The electromechanical equipment is provided with a plurality of camera devices; Accordingly, obtaining a target image of the electromechanical device includes: determining a target camera device among a plurality of camera devices according to the component information; The target image is acquired through the target camera device, and the user is prompted with the location of the target camera device.

5. The method according to any one of claims 1 to 4, characterized in that: The marking the faulty component on the target image according to the component information includes: The outline of the faulty component is marked on the target image according to the component information.

6. A fault detection device, characterized in that: include: A component information acquisition module, used to acquire component information of a faulty component of an electromechanical device, wherein the component information includes location information of the faulty component and feature information of the faulty component, and the electromechanical device is a robot; A target image acquisition module, used to acquire a target image of the electromechanical device; the target image is an image obtained by shooting and containing the faulty component; A target image marking module, used for marking the faulty component on the target image according to the component information, including: marking the faulty component on the target image according to the position information; A target image display module, used for displaying the marked target image; The position information includes relative position information between the faulty component and the characteristic structure, and marking the faulty component on the target image according to the position information includes: Identify the characteristic structure in the target image, and determine the basic coordinates of the target image according to the position of the characteristic structure in the target image; Determine a first position of the faulty component in the target image according to the basic coordinates and the relative position information; Identify the target image according to the first position and the feature information, and determine a second position of the faulty component in the target image; adjusting the first position according to the second position information; The faulty component is marked in the target image according to the first position.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images