Debugging method and device of industrial vision equipment, computer equipment and storage medium

Abstract

The invention relates to a debugging method and device of industrial vision equipment, computer equipment and a storage medium. The method comprises the following steps: receiving a debugging request, wherein the debugging request comprises modal data; performing semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item; generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action; and obtaining a target debugging interface corresponding to the debugging task in the industrial vision equipment, and sending the debugging task to the target debugging interface so as to execute the target debugging action on the target debugging item in the industrial vision equipment. By adopting the method and the device, the industrial vision equipment can be debugged efficiently and accurately.

Description

Technical Field

[0001] The present application relates to the technical field of equipment debugging, and in particular to a debugging method, device, computer equipment and storage medium for industrial vision equipment. Background Art

[0002] In the field of equipment debugging technology, the scenario of deploying and applying industrial vision equipment involves debugging the industrial vision equipment.

[0003] However, for industrial vision equipment, high professional requirements are placed on debugging personnel, and the debugging steps are relatively complicated, which often leads to problems such as low debugging efficiency and unsatisfactory debugging results. Summary of the invention

[0004] Based on this, it is necessary to provide a debugging method, device, computer equipment, computer-readable storage medium and computer program product for industrial vision equipment to address the above technical problems, so as to achieve efficient and accurate debugging of industrial vision equipment.

[0005] In a first aspect, the present application provides a debugging method for an industrial vision device, comprising:

[0006] receiving a debugging request, the debugging request including modal data;

[0007] Performing semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item;

[0008] Generate a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action;

[0009] Obtain a target debugging interface corresponding to the debugging task in the industrial vision device, and send the debugging task to the target debugging interface to perform a target debugging action on the target debugging item in the industrial vision device.

[0010] In a second aspect, the present application provides a debugging device for industrial visual equipment, comprising:

[0011] A receiving module, used for receiving a debugging request, wherein the debugging request includes modal data;

[0012] An analysis module, used to perform semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item;

[0013] A generation module, used for generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action;

[0014] The sending module is used to obtain the target debugging interface corresponding to the debugging task in the industrial vision device, and send the debugging task to the target debugging interface to perform the target debugging action on the target debugging item in the industrial vision device.

[0015] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0017] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps in the above method.

[0018] The above-mentioned debugging method, device, computer equipment, computer-readable storage medium and computer program product of industrial vision equipment, by performing semantic analysis on the modal data in the debugging request, accurately and efficiently obtain the target debugging item, target debugging action and debugging task corresponding to the debugging request, and then send the debugging task to the target debugging interface to execute the target debugging action on the target debugging item in the industrial vision equipment. Based on this, the debugging content of the industrial vision equipment can be adaptively determined according to the modal data in the debugging request, and the industrial vision equipment can be debugged autonomously according to the determined debugging content, thereby reducing the technical support cost for such products as industrial vision equipment, improving product usability, and improving debugging efficiency and debugging accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is an application environment diagram of a debugging method for industrial visual equipment provided by an embodiment of the present application;

[0020] FIG2 is a flow chart of a method for debugging an industrial visual device provided in an embodiment of the present application;

[0021] Figure 3 A structural block diagram of a debugging device for industrial visual equipment provided in an embodiment of the present application;

[0022] Figure 4 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0023] Figure 5 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0024] Figure 6An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] The debugging method of industrial vision equipment provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0027] As shown in FIG2 , the embodiment of the present application provides a debugging method for industrial visual equipment, and the method is applied to the terminal 102 or the server 104 in FIG1 as an example for explanation. It is understandable that the computer device may include at least one of a terminal and a server. The method includes the following steps:

[0028] S202: Receive a debugging request, where the debugging request includes modal data.

[0029] Among them, the debugging request is used to request debugging of industrial vision equipment; industrial vision equipment refers to a series of equipment used for image acquisition, processing and analysis in industrial production and manufacturing processes, such as industrial barcode readers, industrial cameras, intelligent vision sensors and other equipment.

[0030] Among them, modal data refers to data of sensory information types such as vision, hearing, and touch, such as voice data, image data, text data, etc.

[0031] In some embodiments, the modality data in the debugging request may correspond to one type of sensory information or a combination of multiple types of sensory information.

[0032] In some embodiments, based on the hardware configuration information corresponding to the computer device, modal data matching the hardware configuration information can be adaptively applied in the debugging request: for example, if the computer device includes a voice input module for acquiring voice data, such as hardware such as a microphone, then a debugging request containing voice data can be received; if the computer device includes a text input module for acquiring text data, such as hardware such as a display screen and an external keyboard, then a debugging request containing text data can be received; if the computer device includes an image input module for acquiring image data, such as hardware such as a camera and a scanner, then a debugging request containing image data can be received.

[0033] In some embodiments, the debugging request may be received by an industrial vision device; or the debugging request may be received by a designated computer device and sent to the industrial vision device for processing, or the computer device may process the debugging request and send the processed information to the industrial vision device.

[0034] S204: Perform semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item.

[0035] Among them, the debugging item represents the object to be debugged in the industrial vision device, and the debugging action represents the debugging behavior implemented on the object to be debugged; the target debugging item represents the debugging item corresponding to the debugging request, and the target debugging action represents the debugging action corresponding to the debugging request.

[0036] In some embodiments, the debugging item may represent the configuration parameters to be debugged in the industrial vision equipment, for example: the debugging item may represent the image processing parameters of the industrial vision equipment, such as exposure value, gain value, focus value, imaging quality and other parameters; the debugging item may also represent the camera geometric parameters of the industrial vision equipment, such as field of view angle, depth of field range, lens distortion correction and other parameters; the debugging item may also represent the communication configuration parameters of the industrial vision equipment, such as communication protocol, data transmission efficiency, port configuration and other parameters.

[0037] In some embodiments, the debugging item may represent the configuration tools involved in the industrial vision equipment, for example: the debugging item may represent the configuration tool used to record the debugging process of the industrial vision equipment, such as the debugging log; the debugging item may also represent the configuration tools corresponding to the important links in the debugging process of the industrial vision equipment, such as the load testing tool used to implement the performance detection link, the security vulnerability scanning tool used to implement the security detection link, etc.

[0038] In some embodiments, the debugging action may represent debugging behaviors such as adjustment, optimization, and testing performed on a certain debugging item, such as adjusting the exposure value to a specified value, setting the update frequency of the debugging log, etc.; debugging behaviors such as viewing, saving, and exporting performed on a certain debugging item, such as viewing the current value of the exposure value, saving the debugging log, etc.

[0039] In some embodiments, if the modal data is text data, the semantic analysis of the modal data may represent analysis and understanding of the modal data based on natural language processing (NLP) technology, that is, assisting in understanding and interpreting the meaning of the modal data by identifying key words and phrases, identifying grammatical structures and semantics, and converting natural language into a machine-readable format.

[0040] In some embodiments, if the modal data is speech data, the semantic analysis of the modal data may mean assisting in understanding and interpreting the meaning of the modal data by converting speech into text, identifying key words and phrases, identifying the speaker's intentions, identifying the speaker's emotions, etc.

[0041] In some embodiments, if the modal data is image data, the semantic analysis of the modal data may mean assisting in understanding and interpreting the meaning of the modal data by performing semantic segmentation on the image, identifying and locating entities in the image, extracting semantic information of the image, etc.

[0042] S206: Generate a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action.

[0043] The debugging task refers to a task for debugging industrial vision equipment.

[0044] In some embodiments, the debugging task may describe the target debugging items and target debugging actions corresponding to the debugging request in a machine-readable format, that is, define the association between the target debugging items and target debugging actions in the debugging task in a structured data format such as JSON and CSV.

[0045] In some embodiments, each debugging request may correspond to one or more debugging tasks; each debugging task may correspond to a target debugging item and a target debugging action corresponding to the target debugging item, or may correspond to multiple target debugging items and target debugging actions corresponding to each target debugging item.

[0046] S208. Obtain a target debugging interface corresponding to the debugging task in the industrial vision device, and send the debugging task to the target debugging interface to perform a target debugging action on the target debugging item in the industrial vision device.

[0047] The debugging interface refers to a communication channel used to access and interact with data in an industrial vision device to implement debugging of the industrial vision device.

[0048] In some embodiments, different debugging items correspond to different debugging interfaces, and the debugging interface to be sent by the debugging task can be determined according to the target debugging interface corresponding to the target debugging item.

[0049] In some embodiments, debugging items of the same type may correspond to the same debugging interface. For example, debugging items such as exposure value, focus value, etc., which are of the same image processing type, may correspond to the same debugging interface. The debugging interface to be sent for the debugging task may be determined based on the type of the target debugging item.

[0050] In some embodiments, an industrial vision device may receive a debugging request and perform semantic analysis on the modal data in the debugging request to obtain a target debugging item and a target debugging action, generate a corresponding debugging task, and then send the debugging task to a target debugging interface to debug its own configured target debugging item.

[0051] In some embodiments, a computer device may receive a debugging request and perform semantic analysis on the modal data in the debugging request to obtain a target debugging item and a target debugging action, generate a corresponding debugging task, and then establish a communication connection between the computer device and the industrial vision device through a target debugging interface, so as to send the debugging task to the industrial vision device through the target debugging interface and debug the target debugging item configured by the industrial vision device.

[0052] In some embodiments, after receiving a debugging request and before obtaining a response from the industrial vision device, the reception of the next debugging request can be restricted, or the processing of the next debugging request can be restricted; or after receiving a debugging request and before obtaining a response from the industrial vision device, the reception of the next debugging request and processing can be allowed, thereby enabling continuous interaction and dialogue with the user and maintaining context consistency without frequent waiting.

[0053] It can be seen that in the embodiment of the present application, by performing semantic analysis on the modal data in the debugging request, the target debugging item and the target debugging action corresponding to the debugging request are accurately and efficiently obtained, and the debugging task corresponding to the target debugging item and the target debugging action is obtained, and then the debugging task is sent to the target debugging interface to execute the target debugging action on the target debugging item in the industrial vision device. Based on this, the debugging content of the industrial vision device can be adaptively determined according to the modal data in the debugging request, and the industrial vision device can be debugged autonomously according to the determined debugging content, thereby reducing the technical support cost for such products as industrial vision equipment, improving product usability, and improving debugging efficiency and debugging accuracy.

[0054] In some embodiments, generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action includes one of the following two steps:

[0055] S302: Determine the task execution status of the debugging task based on the task execution indicator according to the semantic analysis result corresponding to the preset task execution indicator in the modal data.

[0056] S304: Determine the task execution status of the debugging task based on the task execution indicator according to the reference debugging information corresponding to the preset task execution indicator in the current debugging environment.

[0057] The task execution status of the debugging task indicates the execution status of the debugging task during the execution process; and the task execution index indicates an index for measuring the task execution status of the debugging task.

[0058] In some embodiments, the task execution status of the debugging task can be evaluated by task execution indicators related to time management, such as execution time, response time, and waiting time; the task execution status of the debugging task can also be evaluated by task execution indicators related to resource management, such as CPU usage, memory usage, etc.; the task execution status of the debugging task can also be evaluated by task execution indicators related to task management, such as priority processing, concurrent processing, and queuing processing.

[0059] In some embodiments, in the semantic analysis of modal data, semantic analysis results corresponding to task execution indicators can be extracted based on sentiment analysis, keyword recognition, context association, etc. of the modal data to determine the task execution status corresponding to the debugging task: for example, if it is identified in the modal data that the debugging task needs to be executed at a specified time, it is determined that there is a semantic analysis result corresponding to the task execution indicator at the execution time level, so as to set the task execution status of the debugging task at the execution time level according to the semantic analysis result; for another example, if it is identified in the modal data that the debugging task needs to be executed urgently, it is determined that there is a semantic analysis result corresponding to the task execution indicator at the priority processing level, so as to set the task execution status of the debugging task at the priority processing level according to the semantic analysis result.

[0060] In some embodiments, in the current debugging environment, the task execution status corresponding to the debugging task can be determined based on the reference debugging information corresponding to the task execution indicator, wherein: the current debugging environment can represent the system environment corresponding to the debugging process of the industrial vision device, and the reference debugging information represents the optimal value corresponding to the task execution indicator based on the current system environment. For example, the resource utilization, cost-effectiveness, security, compatibility and other system environments corresponding to the industrial vision device and / or other computer devices involved in the debugging process are analyzed and evaluated, so as to determine the optimal values ​​corresponding to the current system environment for each task execution indicator, so as to determine the task execution status corresponding to the debugging task based on the current system environment.

[0061] In some embodiments, the current debugging environment may represent the user usage environment corresponding to the debugging process of the industrial vision equipment. The reference debugging information indicates that the task execution indicator is based on the optimal value corresponding to the current user usage environment. For example, the debugging preferences and debugging trends of the current user and / or group users, the debugging tendencies of specific industries or application scenarios and other user usage environments are analyzed and evaluated to determine the optimal values ​​of each task execution indicator based on the current user usage environment, so as to determine the task execution status of the debugging task based on the current user usage environment.

[0062] In some embodiments, sending a debugging task to a target debugging interface to perform a target debugging action on a target debugging item in an industrial vision device includes:

[0063] S306: Send the debugging task and the task execution status corresponding to the debugging task to the target debugging interface, so as to perform a target debugging action on the target debugging item in the industrial vision device according to the task execution status.

[0064] In some embodiments, an association relationship may be established between a debugging task and the task execution status corresponding to the debugging task, or the task execution status corresponding to the debugging task may be directly stored in the debugging task, so that the target debugging interface may identify and transmit the debugging task and the corresponding task execution status in an associative and synchronous manner.

[0065] It can be seen that in this embodiment, on the one hand, according to the semantic analysis results corresponding to the task execution indicators in the modal data, it is possible to accurately determine the task execution status corresponding to the debugging task based on the actual debugging needs of the user; on the other hand, according to the reference debugging information corresponding to the task execution indicators in the current debugging environment, it is possible to adaptively determine the task execution status corresponding to the debugging task based on the current debugging environment.

[0066] In some embodiments, a debugging task corresponding to the debugging request is generated according to the target debugging item and the target debugging action, and one of the following two steps is performed:

[0067] S402: If the debugging task does not meet the task execution conditions set based on the industrial vision device, the debugging task is terminated and an alarm message corresponding to the debugging request is generated.

[0068] S404: If the debugging task does not meet the task execution condition set based on the industrial vision device, the expected debugging result corresponding to the debugging task in the current debugging environment is obtained, and the debugging task is updated according to the expected debugging result to obtain an updated debugging task.

[0069] In some embodiments, the task execution condition indicates the conditions that the target debugging item and / or target debugging action must meet when the debugging task is executed. For example, the task execution condition can be used to determine whether the target debugging item is a debugging item corresponding to the debugging of the industrial vision device. For example, when debugging a certain parameter, the parameter is a debugging item associated with the industrial vision device, not a debugging item associated with other computer devices. The task execution condition can be used to determine whether the target debugging action is within the allowable operating range. For example, after adjusting the value of a certain parameter, the adjusted value of the parameter is within the allowable range of values ​​for the parameter. The task execution condition can be used to determine whether there is debugging permission corresponding to the target debugging item and / or target debugging action. For example, when adjusting the value of a certain parameter, the debugging permission to adjust the parameter exists.

[0070] In some embodiments, in the current system environment, the task execution condition may represent the conditions that need to be satisfied by the task execution status of the debugging task, for example: in the current system environment, the task execution condition may be used to determine whether the debugging task can be implemented based on the current system environment based on the task execution status related to time management, such as execution time, response time, and waiting time; in the current system environment, the task execution condition may be used to determine whether the debugging task can be implemented based on the current system environment based on the task execution status related to resource management, such as CPU usage rate, memory usage rate, etc.; in the current system environment, the task execution condition may be used to determine whether the debugging task can be implemented based on the current system environment based on the task execution status related to task management, such as priority processing, concurrent processing, and queuing processing.

[0071] In some embodiments, if the debugging task does not meet the task execution conditions set based on the industrial vision equipment, the debugging task is terminated and an alarm message corresponding to the debugging request is generated. For example, the alarm message can be generated in the form of displaying text on the display screen, playing audio on the speaker, vibrating with a vibration motor, flashing an LED light, etc. to achieve an alarm effect; further, the task execution conditions that are not met by the debugging task can be recorded in the alarm message for reference by the debugging personnel.

[0072] In some embodiments, if the debugging task does not meet the task execution conditions set based on the industrial vision device, the expected debugging result corresponding to the debugging task in the current debugging environment is obtained, and the target debugging item and / or target debugging action in the debugging task is updated according to the expected debugging result: for example, the expected debugging result corresponding to the debugging task can be determined based on the system environment corresponding to the debugging process of the industrial vision device, that is, the expected debugging result can meet the user's debugging needs to the greatest extent under the dimensions of resource utilization, cost-effectiveness, security, compatibility, etc. applicable to the current system environment; for another example, the expected debugging result corresponding to the debugging task can also be determined based on the user usage environment corresponding to the debugging process of the industrial vision device, that is, the expected debugging result can meet the current user's debugging needs to the greatest extent under the dimensions of debugging preferences, debugging trends, etc. applicable to the current user usage environment.

[0073] It can be seen that in this embodiment, if the debugging task does not meet the task execution conditions set based on the industrial vision equipment, on the one hand, the debugging task can be efficiently terminated and an alarm message can be generated. On the other hand, the debugging task can be adaptively updated according to the expected debugging effect corresponding to the debugging task. Based on this, the data applicability and data reliability for debugging the industrial vision equipment are ensured, thereby improving the debugging efficiency.

[0074] In some embodiments, before receiving the debugging request, the method further includes:

[0075] S502: Receive a state switching request.

[0076] The state switching request is used to request to switch the state of the industrial vision device.

[0077] In some embodiments, the status of the industrial vision device includes a working status and a debugging status. The working status indicates the operating status of the industrial vision device in a normal production environment, and the debugging status indicates the special status corresponding to the debugging of the industrial vision device during troubleshooting, performance optimization, function adjustment, etc.

[0078] S504: If the state switching request satisfies the first semantic condition set based on the state switching of the industrial vision device, the industrial vision device is switched to the debugging state, and a semantic analysis permission for performing semantic analysis on the debugging request is obtained.

[0079] S506: If the state switching request satisfies the second semantic condition set based on the state switching of the industrial vision device, the industrial vision device is switched to the working state, and the semantic analysis authority for performing semantic analysis on the debugging request is released.

[0080] Among them, the first semantic condition is used to determine whether the state switching request is a request to switch the industrial vision device to a debugging state; the second semantic condition is used to determine whether the state switching request is a request to switch the industrial vision device to a working state.

[0081] In some embodiments, after receiving a state switching request, a semantic analysis can be performed on the modal data in the state switching request. If the modal data in the state switching request meets a first semantic condition, the industrial vision device is switched to a debugging state. When the industrial vision device is in the debugging state, a semantic analysis permission for performing semantic analysis on the debugging request is obtained to allow semantic analysis of the modal data in the debugging request.

[0082] In some embodiments, if the modal data in the state switching request meets the second semantic condition, the industrial vision device is switched to a working state. When the industrial vision device is in the working state, the semantic analysis permission for performing semantic analysis on the debugging request is released to limit the semantic analysis of the modal data in the debugging request.

[0083] In some embodiments, the first semantic condition or the second semantic condition may be a specified keyword. If the modal data in the state switching request matches the keyword, it means that the state switching request satisfies the first semantic condition or the second semantic condition. For example, the first semantic condition may represent the keyword "start debugging", and the second semantic condition may represent the keyword "end debugging".

[0084] In some embodiments, the first semantic condition or the second semantic condition may be a specified meaning. If the meaning corresponding to the modal data in the state switching request matches the specified meaning, it means that the state switching request satisfies the first semantic condition or the second semantic condition. For example, the first semantic condition may indicate that "start debugging", "start debugging", "start debugging state", etc. are all expressed as the meanings corresponding to the words or phrases for switching to the debugging state. The second semantic condition may indicate that "end debugging", "stop debugging", "start working state", etc. are all expressed as the meanings corresponding to the words or phrases for switching to the working state.

[0085] In some embodiments, when the industrial vision device is in a debugging state, the communication permissions corresponding to each debugging interface can be obtained, so that during the debugging process, the corresponding target debugging interface that has obtained the communication permissions can be called according to the debugging task; when the industrial vision device is in a working state, the communication permissions corresponding to each debugging interface can be released to limit the calling of any debugging interface during normal working.

[0086] It can be seen that in this embodiment, on the one hand, the industrial vision equipment can be efficiently and accurately switched between the working state and the debugging state through the state switching request, and the semantic analysis permissions corresponding to the debugging request can be adaptively processed according to the state of the industrial vision equipment to improve the stability of the industrial vision equipment during the working process and the debugging process.

[0087] In some embodiments, after generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action, the method further includes:

[0088] S602: If the debugging task satisfies the third semantic condition set based on the task type, the debugging task is used as a debugging task to be sent to the target debugging port to obtain a debugging result returned by the industrial vision device through the target debugging port.

[0089] S604: If the debugging task satisfies the fourth semantic condition set based on the task type, the debugging task is used as a debugging task executed based on the current debugging environment, and the debugging task is executed and a debugging result corresponding to the debugging request is generated.

[0090] Among them, the third semantic condition is used to determine whether the debugging task is a task type that needs to be executed by the industrial vision device through the debugging port; the fourth semantic condition is used to determine whether the debugging task is a task type that can be directly executed based on the current debugging environment.

[0091] The debugging result may indicate the information corresponding to the debugging request returned after the debugging task is successfully executed, or the prompt information returned indicating successful execution; or the prompt information returned indicating failure of execution when an exception occurs during the execution of the debugging task.

[0092] In some embodiments, in the semantic analysis of the debugging request, if it is detected that the debugging task belongs to the task type at the control level of the industrial vision device, for example, the debugging task is to perform debugging operations on objects such as configuration parameters and configuration tools in the industrial vision device, then it means that the debugging task satisfies the third semantic condition, and the debugging task is used as a debugging task to be sent to the target debugging port, so as to send the debugging task to the industrial vision device through the target debugging port for execution.

[0093] In some embodiments, in the semantic analysis of the debugging request, if it is detected that the debugging task belongs to the task type at the user interaction level, for example, the debugging task is to specify the data format of the output content, to specify the receiving address of the output content, to specify the display method of the output content, and other operations involving the user interaction level, then it means that the debugging task meets the fourth semantic condition, the debugging task is treated as a debugging task executed based on the current debugging environment, and the debugging task is directly executed based on the current debugging environment.

[0094] It can be seen that in this embodiment, by determining the task type corresponding to the debugging task, the execution path to which the debugging task belongs can be efficiently and accurately determined based on the task type to which the debugging task belongs, so as to improve the execution efficiency of the debugging task.

[0095] In some embodiments, after sending the debugging task to the target debugging interface to perform the target debugging action on the target debugging item in the industrial vision device, the method further includes:

[0096] S702: Receive the debugging result returned through the target debugging interface.

[0097] S704. Convert the debugging result into a target debugging result according to the target data type corresponding to the modal data, where the target debugging result is a debugging result represented by the target data type.

[0098] The data type may represent data representation forms such as audio, text, and image; the target data type represents the data type corresponding to the modal data in the debugging request.

[0099] In some embodiments, the debugging result corresponding to the debugging task can be converted into a target debugging result represented by the target data type according to the target data type corresponding to the modal data in the debugging request. For example, if the modal data in the debugging request is audio data, the debugging result is correspondingly represented by audio data to realize debugging of industrial vision equipment through voice interaction.

[0100] In some embodiments, based on the hardware configuration information corresponding to the computer device, modal data matching the hardware configuration information can be adaptively applied in the debugging results: for example, if the computer device includes an audio output module for outputting audio data, such as hardware such as speakers, then debugging results expressed in the form of audio data can be generated; if the computer device includes a text output module for outputting text data, or includes an image output module for outputting image data, such as hardware such as display screens, projectors, printers, etc., then debugging results expressed in the form of text data can be generated, or debugging results expressed in the form of image data can be generated.

[0101] In some embodiments, if the debugging task belongs to the task type at the user interaction level mentioned above, the debugging result corresponding to the debugging task can also be converted into a target debugging result represented by the target data type in the current debugging environment according to the target data type corresponding to the modal data in the debugging request.

[0102] It can be seen that in this embodiment, the debugging result can be adaptively represented by the target data type according to the target data type corresponding to the modal data in the debugging request to ensure the consistency of the data representation of the input debugging request and the output debugging result.

[0103] In some embodiments, semantic analysis is performed on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item, including:

[0104] S802: If the target debugging item and the target debugging action corresponding to the debugging request are not identified according to the semantic analysis of the modal data, an update request is returned, where the update request is used to request an update of the debugging request.

[0105] In some embodiments, the update request can be returned to the computer device corresponding to the debugging request, which is equivalent to, if the industrial vision device receives a debugging request from a computer device, the industrial vision device returns the update request to the computer device; if the first computing device receives a debugging request from a second computer device, the first computer device returns the update request to the second computer device.

[0106] In some embodiments, the data representation of the update request may match the data representation of the debug request, for example: if the debug request is represented by voice data, the update request may be correspondingly represented by voice data; if the debug request is represented by text data, the update request may be correspondingly represented by text data.

[0107] In some embodiments, the update request is used to provide suggestions to the user at the level of semantic correction to guide the user to update the debugging request, for example: in the grammatical dimension, optimizing vocabulary and sentence structure, correcting grammatical errors and improving expressions, improving the accuracy and professionalism of the language, etc.; in the content dimension, eliminating ambiguity, adding contextual information, supplementing necessary background and details, etc.; in the structural dimension, adding conjunctions, using transition sentences, highlighting key points, etc.

[0108] In some embodiments, the update request is used to provide suggestions to the user at the user preference level to guide the user to update the debugging request. For example, based on the debugging habits of the current user or the debugging tendencies of group users, the user is guided to update the debugging request based on common debugging items and debugging actions in the debugging process.

[0109] S804: Receive an updated debugging request corresponding to the update request, and re-perform semantic analysis on the modal data in the updated debugging request until a target debugging item and a target debugging action corresponding to the updated debugging request are identified.

[0110] In some embodiments, after receiving an updated debugging request, semantic analysis is performed again on the modal data in the updated debugging request. If the target debugging item and target debugging action corresponding to the updated debugging request are still not identified, a new update request is returned and a new debugging request is received until the target debugging item and target debugging action corresponding to the debugging request are identified.

[0111] In some embodiments, if the update request is used to provide suggestions to the user at the level of semantic correction, the update request generated in a new round corresponds to a deeper level of semantic correction, for example: the first round of update requests is used to correct spelling errors, the second round of update requests is used to adjust sentence structure, the third round of update requests is used to add transition words to improve fluency, and the fourth round of update requests is used to focus on the optimization of tone and overall semantics. Based on the above-mentioned progressive semantic correction method, users can gradually improve the quality of expression while avoiding giving too many complex suggestions at one time to cause trouble to users.

[0112] It can be seen that in this embodiment, if the corresponding target debugging item and target debugging action are not identified based on the semantic analysis of the modal data in the debugging request, an update request is returned to efficiently guide the user to adaptively update the debugging request through multiple rounds of interaction with the user, so as to ensure that the target debugging item and target debugging action corresponding to the updated debugging request can be identified, thereby ensuring the accuracy and reliability of the semantic analysis result.

[0113] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps. Based on the same inventive concept, the embodiment of the present application also provides a debugging device for industrial visual equipment. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific definition in the debugging device embodiment of one or more industrial visual equipment provided below can refer to the definition of the debugging method for industrial visual equipment in the above text, which will not be repeated here.

[0114] like Figure 3 As shown, the embodiment of the present application provides a device 300, including:

[0115] The receiving module 302 is used to receive a debugging request, where the debugging request includes modal data;

[0116] The analysis module 304 is used to perform semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item;

[0117] A generating module 306, configured to generate a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action;

[0118] The sending module 308 is used to obtain the target debugging interface corresponding to the debugging task in the industrial vision device, and send the debugging task to the target debugging interface to perform the target debugging action on the target debugging item in the industrial vision device.

[0119] In some embodiments, in terms of generating a debugging task corresponding to a debugging request according to a target debugging item and a target debugging action, the generating module 306 is specifically configured to:

[0120] Determine the task execution status of the debugging task based on the task execution indicator according to the semantic analysis result corresponding to the preset task execution indicator in the modal data;

[0121] Determine the task execution status of the debugging task based on the task execution indicator according to the reference debugging information corresponding to the preset task execution indicator in the current debugging environment;

[0122] In some embodiments, in sending the debugging task to the target debugging interface to perform the target debugging action on the target debugging item in the industrial vision device, the sending module 308 is specifically used to:

[0123] The debugging task and the task execution status corresponding to the debugging task are sent to the target debugging interface, so as to perform the target debugging action on the target debugging item in the industrial vision device according to the task execution status.

[0124] In some embodiments, in terms of generating a debugging task corresponding to a debugging request according to a target debugging item and a target debugging action, the generating module 306 is specifically configured to:

[0125] If the debugging task does not meet the task execution conditions set based on the industrial vision device, the debugging task is terminated and an alarm message corresponding to the debugging request is generated;

[0126] If the debugging task does not meet the task execution condition set based on the industrial vision device, the expected debugging result corresponding to the debugging task in the current debugging environment is obtained, and the debugging task is updated according to the expected debugging result to obtain an updated debugging task.

[0127] In some embodiments, in terms of state switching of the industrial vision device, the apparatus 300 further includes a state switching module, which is specifically used to:

[0128] Receive a state switching request;

[0129] If the state switching request satisfies the first semantic condition set based on the state switching of the industrial vision device, the industrial vision device is switched to the debugging state, and a semantic analysis permission for performing semantic analysis on the debugging request is obtained;

[0130] If the state switching request satisfies the second semantic condition set based on the state switching of the industrial vision device, the industrial vision device is switched to the working state, and the semantic analysis authority for performing semantic analysis on the debugging request is released.

[0131] In some embodiments, in terms of classifying the debugging tasks, the apparatus 300 further includes a task classification module, which is specifically used to:

[0132] If the debugging task satisfies the third semantic condition set based on the task type, the debugging task is used as a debugging task to be sent to the target debugging port, so as to obtain the debugging result returned by the industrial vision device through the target debugging port;

[0133] If the debugging task satisfies the fourth semantic condition set based on the task type, the debugging task is used as a debugging task executed based on the current debugging environment, and the debugging task is executed and a debugging result corresponding to the debugging request is generated.

[0134] In some embodiments, in terms of receiving the debugging result, the apparatus 300 further includes a data conversion module, which is specifically used to:

[0135] Receive debugging results returned through the target debugging interface;

[0136] The debugging result is converted into a target debugging result according to the target data type corresponding to the modal data, and the target debugging result is a debugging result represented by the target data type.

[0137] In some embodiments, in performing semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item, the apparatus 300 further includes an updating module, which is specifically used to:

[0138] If the target debugging item and target debugging action corresponding to the debugging request are not identified according to the semantic analysis of the modal data, an update request is returned, and the update request is used to request to update the debugging request;

[0139] An updated debugging request corresponding to the update request is received, and the modal data in the updated debugging request is semantically analyzed again until a target debugging item and a target debugging action corresponding to the updated debugging request are identified.

[0140] Each module in the debugging device of the above industrial visual equipment can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0141] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data for debugging industrial visual equipment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned method for debugging industrial visual equipment are implemented.

[0142] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the debugging method of the above-mentioned industrial visual equipment are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0143] Those skilled in the art will understand that Figure 4 or Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0145] In some embodiments, Figure 6 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0146] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A debugging method for industrial visual equipment, characterized in that: include: receiving a debugging request, wherein the debugging request includes modality data; Performing semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item; generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action; A target debugging interface corresponding to the debugging task in the industrial vision device is obtained, and the debugging task is sent to the target debugging interface to execute the target debugging action on the target debugging item in the industrial vision device.

2. The method according to claim 1, characterized in that: Generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action includes one of the following two steps: Determine, according to a semantic analysis result corresponding to a preset task execution indicator in the modal data, a task execution state of the debugging task based on the task execution indicator; Determining the task execution status of the debugging task based on the task execution indicator according to the reference debugging information corresponding to the preset task execution indicator in the current debugging environment; The step of sending the debugging task to the target debugging interface to execute the target debugging action on the target debugging item in the industrial vision device includes: The debugging task and the task execution status corresponding to the debugging task are sent to the target debugging interface, so as to execute the target debugging action on the target debugging item in the industrial vision device according to the task execution status.

3. The method according to claim 1, characterized in that Generating a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action includes one of the following two steps: If the debugging task does not meet the task execution condition set based on the industrial vision device, the debugging task is terminated and an alarm message corresponding to the debugging request is generated; If the debugging task does not meet the task execution condition set based on the industrial vision device, the expected debugging result corresponding to the debugging task in the current debugging environment is obtained, and the debugging task is updated according to the expected debugging result to obtain an updated debugging task.

4. The method according to claim 1, characterized in that Before receiving the debugging request, the method further includes: Receive a state switching request; If the state switching request satisfies a first semantic condition set based on the state switching of the industrial vision device, the industrial vision device is switched to a debugging state, and a semantic analysis permission for performing semantic analysis on the debugging request is obtained; If the state switching request satisfies the second semantic condition set based on the state switching of the industrial vision device, the industrial vision device is switched to a working state, and the semantic analysis authority for performing semantic analysis on the debugging request is released.

5. The method according to claim 1, characterized in that: After generating the debugging task corresponding to the debugging request according to the target debugging item and the target debugging action, the method further includes: If the debugging task satisfies the third semantic condition set based on the task type, the debugging task is used as a debugging task to be sent to the target debugging port to obtain the debugging result returned by the industrial vision device through the target debugging port; If the debugging task satisfies the fourth semantic condition set based on the task type, the debugging task is used as a debugging task executed based on the current debugging environment, and the debugging task is executed and a debugging result corresponding to the debugging request is generated.

6. The method according to claim 1, characterized in that After sending the debugging task to the target debugging interface to perform the target debugging action on the target debugging item in the industrial vision device, the method further includes: Receiving the debugging result returned through the target debugging interface; The debugging result is converted into a target debugging result according to a target data type corresponding to the modal data, wherein the target debugging result is a debugging result represented by the target data type.

7. The method according to claim 1, characterized in that The performing semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item includes: If the target debugging item and the target debugging action corresponding to the debugging request are not identified according to the semantic analysis of the modal data, an update request is returned, where the update request is used to request an update of the debugging request; An updated debugging request corresponding to the update request is received, and the modal data in the updated debugging request is semantically analyzed again until a target debugging item and a target debugging action corresponding to the updated debugging request are identified.

8. A debugging device for industrial visual equipment, characterized in that: include: A receiving module, used for receiving a debugging request, wherein the debugging request includes modal data; An analysis module, configured to perform semantic analysis on the modal data to obtain a target debugging item corresponding to the debugging request and a target debugging action corresponding to the target debugging item; A generating module, configured to generate a debugging task corresponding to the debugging request according to the target debugging item and the target debugging action; The sending module is used to obtain the target debugging interface corresponding to the debugging task in the industrial vision device, and send the debugging task to the target debugging interface to execute the target debugging action on the target debugging item in the industrial vision device.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images