Data processing method and device based on industrial quality inspection and electronic equipment

Abstract

The invention discloses a data processing method and device based on industrial quality inspection and electronic equipment. Relates to the technical field of computers. The method comprises the following steps: in a process of synchronizing industrial quality inspection data in a target machine to a storage server, collecting resource occupation data of the target machine, and obtaining error reporting information returned by the storage server; determining operation state information of the target machine based on the resource occupation data and the error reporting information; judging whether the running state information meets a preset condition or not to obtain a judgment result; when the judgment result indicates that the operation state information meets the preset condition, alarm information is generated, and the alarm information is used for indicating that the process of synchronizing the industrial quality inspection data in the target machine to the storage server is abnormal. The technical problem that the abnormal condition of the machine cannot be accurately alarmed due to the fact that the state of the machine is difficult to accurately monitor when the industrial quality inspection data of the machine is synchronized in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, device and electronic equipment based on industrial quality inspection. Background Art

[0002] With the advent of the big data era, the amount of new data added by manufacturing companies every day mostly reaches TB or even PB level. The new data is stored in the database and obtained by querying the database. However, when the database storage capacity reaches a certain level, the query and storage performance will be greatly reduced. On this basis, the machines in the manufacturing industry are prone to abnormal conditions. However, in the relevant technology, only when the machine has obvious abnormalities can it be known that the machine has abnormalities, that is, whether customers and after-sales personnel can perceive the risks in time, which is prone to production accidents.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a data processing method, device and electronic device based on industrial quality inspection, so as to at least solve the technical problem in the related technology that it is difficult to accurately monitor the status of the machine when synchronizing the industrial quality inspection data of the machine, resulting in the inability to accurately warn of abnormal conditions of the machine.

[0005] According to one aspect of an embodiment of the present invention, there is provided a data processing method based on industrial quality inspection, comprising: in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, collecting the resource occupancy data of the target machine, and obtaining the error information returned by the storage server, wherein the error type of the error information comprises at least one of the following: denying the target machine access to the storage server, failure of the target machine to connect to the storage server, and disconnection of the target machine; determining the operating status information of the target machine based on the resource occupancy data and the error information; judging whether the operating status information satisfies a preset condition and obtaining a judgment result; and generating an alarm message when the judgment result indicates that the operating status information satisfies the preset condition, wherein the alarm message is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server.

[0006] Furthermore, collecting resource occupancy data of the target machine includes: determining a target monitoring tool, wherein the target monitoring tool includes: a tool based on a cross-platform library and a command line tool, the cross-platform library is used to collect running processes and resource utilization data in the target machine; using the target monitoring tool to collect data of the target machine to obtain the resource occupancy data.

[0007] Furthermore, based on the resource occupancy data and the error information, the operating status information of the target machine is determined, including: according to a preset rule, setting the status value of the error information to obtain a target status value, wherein the status value is used to indicate the error type of the error information; based on the resource occupancy data and the target status value, obtaining the operating status information.

[0008] Furthermore, the step of synchronizing the industrial quality inspection data in the target machine to the storage server includes: after receiving the file synchronization task issued by the target server, based on the file sharing protocol, reading the address information of the file to be synchronized in the task queue to obtain the target address information, wherein the file synchronization task includes: a synchronization task for synchronizing the industrial quality inspection data in the target machine to the storage server, the task queue is used to store the address information of the file to be synchronized to the storage server in the target machine; based on the address information, reading the file to be synchronized, and writing the read file to be synchronized to the storage server.

[0009] Furthermore, based on the file sharing protocol, the address information of the files to be synchronized in the task queue is read to obtain the target address information, including: obtaining the number of target threads associated with the target machine, wherein the types of the target threads include: lightweight threads; based on the number of target threads, the address information of the files to be synchronized in the task queue is concurrently read to obtain the target address information.

[0010] Furthermore, the read file to be synchronized is written into the storage server, including: when there are pictures in the file to be synchronized, the pictures in the file to be synchronized are transcoded and compressed to obtain a processed file to be synchronized; index information of the processed file to be synchronized is established, and the index information and the processed file to be synchronized are written into the storage server.

[0011] Furthermore, the target server includes: a server built on a container scheduling platform, the target server includes: multiple groups of containers, the multiple groups of containers are used to execute the step of synchronizing the industrial quality inspection data in the target machine to the storage server, and after receiving the file synchronization task issued by the target server, it also includes: when any group of the containers in the target server is in an abnormal state, a new group of containers is started through the target server to replace the containers in the abnormal state.

[0012] According to another aspect of an embodiment of the present invention, there is also provided a data processing device based on industrial quality inspection, including: a processing unit, for collecting resource occupancy data of the target machine and obtaining error information returned by the storage server during the process of synchronizing the industrial quality inspection data in the target machine to the storage server, wherein the error type of the error information includes at least one of the following: denying the target machine access to the storage server, failure of the target machine to connect to the storage server, and disconnection of the target machine; a determination unit, for determining the operating status information of the target machine based on the resource occupancy data and the error information; a judgment unit, for judging whether the operating status information meets a preset condition and obtaining a judgment result; a generation unit, for generating an alarm message when the judgment result indicates that the operating status information meets the preset condition, wherein the alarm message is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server.

[0013] Furthermore, the processing unit includes: a determination subunit, used to determine a target monitoring tool, wherein the target monitoring tool includes: a tool based on a cross-platform library and a command line tool, and the cross-platform library is used to collect the running processes and resource utilization data in the target machine; a collection subunit, used to use the target monitoring tool to collect data of the target machine to obtain the resource occupancy data.

[0014] Furthermore, the determination unit includes: a setting subunit, used to set the status value of the error information according to a preset rule to obtain a target status value, wherein the status value is used to indicate the error type of the error information; and a processing unit, used to obtain the operating status information based on the resource occupancy data and the target status value.

[0015] Furthermore, the processing unit also includes: a reading subunit, which is used to read the address information of the file to be synchronized in the task queue based on the file sharing protocol after receiving the file synchronization task issued by the target server, and obtain the target address information, wherein the file synchronization task includes: a synchronization task of synchronizing the industrial quality inspection data in the target machine to the storage server, and the task queue is used to store the address information of the file to be synchronized to the storage server in the target machine; the processing subunit is used to read the file to be synchronized based on the address information, and write the read file to be synchronized to the storage server.

[0016] Furthermore, the reading sub-unit includes: an acquisition module, used to obtain the number of target threads associated with the target machine, wherein the types of the target threads include: lightweight threads; a reading module, used to concurrently read the address information of the files to be synchronized in the task queue based on the number of target threads to obtain the target address information.

[0017] Furthermore, the processing sub-unit includes: a processing module, which is used to transcode and compress the pictures in the file to be synchronized to obtain the processed file to be synchronized when there are pictures in the file to be synchronized; a processing module, which is used to establish index information of the processed file to be synchronized, and write the index information and the processed file to be synchronized to the storage server.

[0018] Furthermore, the target server includes: a server built on a container scheduling platform, the target server includes: multiple groups of containers, the multiple groups of containers are used to execute the step of synchronizing the industrial quality inspection data in the target machine to the storage server, and the processing unit also includes: a startup subunit, which is used to start a new group of containers through the target server to replace the containers in the abnormal state after receiving the file synchronization task issued by the target server, when any group of the containers in the target server is in an abnormal state.

[0019] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned data processing methods based on industrial quality inspection by executing the executable instructions.

[0020] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned data processing methods based on industrial quality inspection.

[0021] In the present invention, in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, the resource occupancy data of the target machine is collected, and the error information returned by the storage server is obtained, wherein the error type of the error information includes at least one of the following: the target machine is denied access to the storage server, the target machine fails to connect to the storage server, and the target machine is offline; based on the resource occupancy data and the error information, the operating status information of the target machine is determined; it is determined whether the operating status information meets the preset conditions to obtain a judgment result; if the judgment result indicates that the operating status information meets the preset conditions, an alarm information is generated, wherein the alarm information is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server. This solves the technical problem in the related art that it is difficult to accurately monitor the status of the machine when synchronizing the industrial quality inspection data of the machine, resulting in the inability to accurately alarm the abnormal situation of the machine.

[0022] In the present invention, in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, timely alarms are given for abnormal conditions of the machine based on the resource occupancy of the target machine and the error information returned by the storage server, thereby avoiding the situation in the related technology where the operating status of the machine cannot be monitored from multiple dimensions in a timely manner, thereby achieving the technical effect of improving the efficiency of alarms for machine abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1 is a flow chart of an optional data processing method based on industrial quality inspection according to an embodiment of the present invention;

[0025] Figure 2 is an optional data processing flow chart based on industrial quality inspection according to an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of an optional data processing device based on industrial quality inspection according to an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.), collected information and data (including but not limited to data used for analysis, stored data, displayed data, industrial quality inspection data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] For the convenience of description, some terms or nouns involved in the present invention are explained below.

[0032] Goroutine is a lightweight thread in Go (a programming language) that can execute multiple functions or methods concurrently in a program.

[0033] Channel is a communication mechanism between goroutines, which can be used to transfer data between goroutines.

[0034] SMB, a file sharing protocol, is a network communications protocol used to share files, printers, and other resources between computers.

[0035] go-smb refers to a tool or library written in Go language to access the SMB protocol. With go-smb, users can easily access and operate SMB shared resources in Go programs.

[0036] Embodiment 1

[0037] According to an embodiment of the present invention, an optional method embodiment of a data processing method based on industrial quality inspection is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] Figure 1 is a flowchart of an optional data processing method based on industrial quality inspection according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0039] Step S101, in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, the resource usage data of the target machine is collected, and the error information returned by the storage server is obtained, wherein the error type of the error information includes at least one of the following: the target machine is denied access to the storage server, the target machine fails to connect to the storage server, and the target machine is offline.

[0040] The above-mentioned target machines may include: machines related to industrial quality inspection in the manufacturing industry (referred to as Agent machines), the above-mentioned resource usage data may include but are not limited to: CPU (Central Processing Unit), GPU (Graphics Processing Unit), memory, disk usage, bandwidth and other data, the above-mentioned error information may include but is not limited to: denial of target machine access to the storage server, failure of the target machine to connect to the storage server and target machine disconnection, etc. The above-mentioned storage server storage data method may include: object storage and SMB (a file sharing protocol) shared storage.

[0041] In this embodiment, the resource usage data of the local disk of the target machine can be collected through a monitoring tool (or a surveillance tool), and the error information returned by the storage server can also be collected.

[0042] Step S102, determining the operating status information of the target machine based on the resource occupancy data and the error information.

[0043] In this embodiment, the resource usage data and the returned error information may constitute the operation status information (HealthStatus) of the target machine.

[0044] Step S103, determining whether the running status information meets a preset condition, and obtaining a determination result.

[0045] The above-mentioned preset conditions may include: a condition that the target machine is abnormal. In order to timely determine whether the target machine is abnormal, it is also possible to determine whether the operating status information meets the preset conditions to obtain a determination result.

[0046] Step S104, when the judgment result indicates that the operation status information meets the preset condition, an alarm message is generated, wherein the alarm message is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server.

[0047] When the judgment result indicates that the operating status information meets the preset conditions, an alarm message can be generated to prompt that there is an abnormality in the process of synchronizing the industrial quality inspection data to the storage server of the target machine. The alarm message can also include: abnormal data that occurs during the operation of the target machine.

[0048] In an optional example, the process of synchronizing the industrial quality inspection data in the target machine to the storage server can also be monitored by a third monitoring platform. For example, the HTTP (Hypertext Transfer Protocol) proxy function can be used to provide a metrics interface (monitoring interface for indicator data) on the server side (the server side that handles the monitoring information of the process of monitoring the synchronization of industrial quality inspection data to the storage server). The third-party monitoring platform actively pulls and determines whether to generate an alarm message based on the conditions of the trigger. Alarms can also be issued using email or other office software.

[0049] It should be noted that, in this embodiment, for generating alarm information, a metrics interface + a custom monitoring template can be provided on the server side.

[0050] In this embodiment, through the above steps, in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, based on the resource occupancy of the target machine and the error information returned by the storage server, the abnormal situation of the machine can be promptly warned, avoiding the situation in the related art that the operating status of the machine cannot be monitored from multiple dimensions in a timely manner, thereby achieving the technical effect of improving the efficiency of warning of abnormal machine. In addition, the technical problem that it is difficult to accurately monitor the status of the machine when synchronizing the industrial quality inspection data of the machine in the related art, resulting in the inability to accurately warn of the abnormal situation of the machine, is solved.

[0051] Optionally, collecting resource usage data of the target machine includes: determining a target monitoring tool, wherein the target monitoring tool includes: a tool based on a cross-platform library and a command line tool, the cross-platform library is used to collect running processes and resource utilization data in the target machine; using the target monitoring tool to collect data of the target machine to obtain resource usage data.

[0052] The above-mentioned target monitoring tool can be a tool composed of a cross-platform library and a command line tool. The cross-platform library can refer to a cross-platform Python (a programming language) library for retrieving information about system processes and system utilization. The command line tool can be a tool for synchronizing and backing up files. In this embodiment, the cross-platform library can be used to obtain the system resource information of the local disk of the target machine, and then the command line tool can be used to back up this information to obtain the above-mentioned resource usage data.

[0053] For example, the cross-platform library of the golang (a programming language) class library can be used to collect information about the local disk and CPU of the target machine. The bandwidth usage of the machine can be returned through the rclone API to return resource usage data, achieving the purpose of multi-dimensional and fine-grained monitoring of the data synchronization process of the target machine and real-time monitoring of the machine status.

[0054] Optionally, based on the resource occupancy data and the error information, the operating status information of the target machine is determined, including: setting the status value of the error information according to a preset rule to obtain a target status value, wherein the status value is used to indicate the error type of the error information; and obtaining the operating status information based on the resource occupancy data and the target status value.

[0055] In this embodiment, the corresponding status value is set according to the error information returned by the remote end (corresponding to the storage server). For example, if AWS API (a set of application program interfaces for storage servers) returns 403, it means that access is denied; if the storage connection through AWS API fails, it means that the target storage is unreachable; if the server (corresponding to the storage server) detects that the machine is not connected, it will alarm that the machine is offline.

[0056] In this embodiment, the HealthStatus (corresponding to the running status information) of the target machine can be composed of the resource occupancy data and the target status value indicating the error information, thereby achieving the purpose of comprehensively monitoring the status of the target machine.

[0057] Optionally, the step of synchronizing the industrial quality inspection data in the target machine to the storage server includes: after receiving the file synchronization task issued by the target server, based on the file sharing protocol, reading the address information of the file to be synchronized in the task queue, and obtaining the target address information, wherein the file synchronization task includes: a synchronization task for synchronizing the industrial quality inspection data in the target machine to the storage server, the task queue is used to store the address information of the file to be synchronized to the storage server in the target machine; based on the address information, reading the file to be synchronized, and writing the read file to be synchronized to the storage server.

[0058] In this embodiment, the industrial quality inspection data in the target machine can be synchronized to the storage server according to goroutine+channel+storage protocol (file sharing protocol).

[0059] For example, a task queue can be maintained locally on the target machine, in which the file path (corresponding to address information) of the data that needs to be synchronized (i.e., the files to be synchronized) can be stored. The data in the task queue can be consumed concurrently, that is, the address information of each file to be synchronized can be obtained, and then the file to be synchronized can be read according to the address information to obtain the file stream, and then the file to be synchronized can be written to the storage server according to the file stream.

[0060] For example, based on the task queue, the file sharing protocol can be used to access and obtain the file stream corresponding to the file to be synchronized. For example, for SMBV1.0 (file sharing protocol version 1.0) and SMBV2.1 (file sharing protocol version 2.1), the SMB client is implemented according to go-smb, and then the corresponding file to be synchronized is read and written to the target end (i.e., the storage server).

[0061] According to the file sharing protocol, the files to be synchronized are written to the storage server, thereby achieving the purpose of improving the compatibility of synchronizing industrial quality inspection data to the storage server.

[0062] Optionally, based on the file sharing protocol, the address information of the files to be synchronized in the task queue is read to obtain the target address information, including: obtaining the number of target threads associated with the target machine, wherein the types of target threads include: lightweight threads; based on the number of target threads, the address information of the files to be synchronized in the task queue is concurrently read to obtain the target address information.

[0063] In this embodiment, the associated to-be-synchronized files in the channel (corresponding to the task queue) can also be concurrently consumed according to the number of goroutines (i.e., the number of lightweight threads) configured on the target machine. The address information of the to-be-synchronized files in the task queue can be concurrently read based on the number of target threads to obtain the target address information.

[0064] In an optional example, for some large batches of files, the customer can appropriately adjust the concurrency of the target machine, that is, adjust the number of target threads, according to the configuration of their own machine.

[0065] By concurrently consuming the task queue by the number of target threads, the technical effect of improving the synchronization efficiency of industrial quality inspection data to the storage server is achieved.

[0066] Optionally, the read file to be synchronized is written into a storage server, including: when there are pictures in the file to be synchronized, transcoding and compressing the pictures in the file to be synchronized to obtain the processed file to be synchronized; establishing index information of the processed file to be synchronized, and writing the index information and the processed file to be synchronized into the storage server.

[0067] In this embodiment, the image data in the synchronization file can be transcoded and compressed using the image processing library that comes with golang. The corresponding plug-in can be implemented on the target machine side. After the data synchronization task is delegated and before the data is synchronized, the data to be synchronized can be processed twice. For example: image transcoding and compression uses golang's built-in jpg, png and other libraries to decode and encode the images to implement the transcoding function (i.e., transcoding and compressing the images in the synchronization file). Document tag function: read the configuration file stream, parse the corresponding configuration, and then add the corresponding tag information before writing to the remote end (i.e., establish the index information of the processed file to be synchronized).

[0068] Secondary processing of synchronized files: According to different needs of customers, secondary processing can be performed through plug-ins during collection. For example, image transcoding and compression plug-ins and document tag plug-ins can be provided to transcode and compress images in synchronized files and create tags related to index information.

[0069] It should be noted that in this embodiment, according to the plug-in processing capability of the machine, various business scenarios of customers can be met, and customers can choose different plug-ins for processing according to their needs. For example, if the customer's target machine has a 400M jpg image locally, and the customer wants to compress the image before uploading the data synchronously, then the image transcoding compression plug-in can be selected. The technical effect of reducing the occupancy rate of the image on the storage space is achieved.

[0070] Optionally, the target server includes: a server built on a container scheduling platform, the target server includes: multiple groups of containers, the multiple groups of containers are used to execute the step of synchronizing the industrial quality inspection data in the target machine to the storage server, after receiving the file synchronization task issued by the target server, it also includes: when any group of containers in the target server is in an abnormal state, a new group of containers is started through the target server to replace the containers in the abnormal state.

[0071] Before receiving the file synchronization task sent by the target server, you can also use the container scheduling platform to build a server to send synchronization plan tasks to the target machine, so that the target machine can collect industrial quality inspection data and synchronize it to the corresponding storage server.

[0072] According to the cluster scheduling of the container scheduling platform, elastic expansion and contraction are achieved, and load balancing is achieved through the container scheduling platform. When a pod (a group of containers) is in an abnormal state (for example, it is down due to an abnormality), the container scheduling platform can re-pull a new pod (corresponding to a new group of containers) to ensure the overall stability and high availability of the service. Reliability and stability, fast elastic expansion and contraction.

[0073] In an optional example, after the collected industrial quality inspection data is synchronized to the storage server, a visualization page can be provided to allow customers to implement advanced retrieval functions by enabling metadata queries (establishing corresponding indexes in the distributed search engine on the server side).

[0074] Figure 2 is an optional data processing flow chart based on industrial quality inspection according to an embodiment of the present invention, such as Figure 2 As shown, including:

[0075] 1. Configure input source: You can collect log files through the agent and push them to the logserver (log service). The logserver can obtain changelog (logs recording data changes) through the api (application programming interface). The logserver can obtain object data through S3. The logserver can also obtain other data sources (such as Figure 2 RDB (relational database) in.

[0076] 2. The parsing module processes the collected data: parse, transform and filter (etl), sink (write) the processed data to ES (search server), regularly agg (aggregate) the business data and clean up historical data.

[0077] 3. Provide a data query interface to the outside world, directly search ES, and query agg data.

[0078] in, Figure 3 In the above code, pg and redis represent storage space for data synchronization tasks, S3 can represent storage space for industrial quality inspection data, and configuration management can be used to manage data synchronization tasks, such as synchronizing data from one location to another, how often to synchronize data, etc. contrl can represent control, Console can refer to the console, and sdk can represent the software development kit.

[0079] The data processing method based on industrial quality inspection provided by this embodiment can provide the following functions: (1) Secondary processing function: according to different customer needs, secondary processing is performed through plug-ins during collection, for example, image transcoding and compression plug-ins and document tag plug-ins are provided; (2) Monitoring function: multi-dimensional monitoring of client station conditions, including but not limited to: CPU, GPU, memory, disk usage, bandwidth, to ensure the stability and reliability of customer services; (3) Alarm function: support multi-dimensional and multi-path fine-grained alarms, and support docking with third-party monitoring platforms. Alarms include but are not limited to: Agent machine offline, current storage unreachable, clock asynchronization, machine IP conflict, label name duplication, storage access denied, so that when the target machine is abnormal, customers and after-sales personnel can perceive the risk in time and make corresponding treatment, thus nipping the production accident in the bud; (4) Performance and compatibility functions: In addition to supporting the collection of local data of the machine, it can also collect data from shared storage such as SMB, including but not limited to: Local (local of the target machine) -> S3 (object storage), Local -> SMB, S3 -> S3, S3 -> SMB, SMB -> S3, SMB -> SMB. In terms of performance, the number of concurrent Agent machines can also be set to improve the data collection capability; (5) Reliability and stability: Rapid elastic expansion and contraction: Deploy the service on the container scheduling platform, and the container scheduling platform will schedule the service. When the load of a service pod is too high, the deployment can be adjusted to achieve rapid expansion, and the load of the pod can be reduced through the load balancing mechanism. When the pod is down due to an accident, the container scheduling platform can actively try to restart a new pod to provide external services. Ensure the stability of external services.

[0080] Embodiment 2

[0081] The second embodiment of the present invention provides an optional data processing device based on industrial quality inspection, and each implementation unit in the data processing device corresponds to each implementation step in the first embodiment.

[0082] Figure 3 is a schematic diagram of an optional data processing device based on industrial quality inspection according to an embodiment of the present invention, such as Figure 3 As shown, the data processing device includes: a processing unit 31, a determining unit 32, a judging unit 33 and a generating unit 34.

[0083] The processing unit 31 is used to collect resource usage data of the target machine and obtain error information returned by the storage server during the process of synchronizing the industrial quality inspection data in the target machine to the storage server, wherein the error type of the error information includes at least one of the following: denying the target machine access to the storage server, failure of the target machine to connect to the storage server, and disconnection of the target machine;

[0084] A determination unit 32, configured to determine the operation status information of the target machine based on the resource occupancy data and the error information;

[0085] The judging unit 33 is used to judge whether the running status information meets the preset conditions and obtain the judging result;

[0086] The generating unit 34 is used to generate an alarm message when the judgment result indicates that the operation status information meets a preset condition, wherein the alarm message is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server.

[0087] In the data processing device based on industrial quality inspection provided in the second embodiment of the present invention, the processing unit 31 can be used to collect the resource occupancy data of the target machine and obtain the error information returned by the storage server during the process of synchronizing the industrial quality inspection data in the target machine to the storage server, wherein the error type of the error information includes at least one of the following: the target machine is denied access to the storage server, the target machine fails to connect to the storage server, and the target machine is offline. The determination unit 32 determines the operation status information of the target machine based on the resource occupancy data and the error information; the judgment unit 33 is used to judge whether the operation status information meets the preset conditions and obtain the judgment result. The generation unit 34 generates an alarm information when the judgment result indicates that the operation status information meets the preset conditions, wherein the alarm information is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server. This solves the technical problem in the related art that it is difficult to accurately monitor the status of the machine when synchronizing the industrial quality inspection data of the machine, resulting in the inability to accurately alarm the abnormal situation of the machine.

[0088] In this embodiment, in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, timely alarms are issued for abnormal conditions of the machine based on the resource occupancy of the target machine and the error information returned by the storage server, thereby avoiding the situation in related technologies where the operating status of the machine cannot be monitored from multiple dimensions in a timely manner, thereby achieving the technical effect of improving the efficiency of alarms for machine abnormalities.

[0089] Optionally, in the data processing device based on industrial quality inspection provided in Example 2 of the present invention, the processing unit includes: a determination subunit, used to determine a target monitoring tool, wherein the target monitoring tool includes: a tool based on a cross-platform library and a command line tool, the cross-platform library is used to collect running processes and resource utilization data in the target machine; a collection subunit, used to use the target monitoring tool to collect data on the target machine to obtain resource occupancy data.

[0090] Optionally, in the data processing device based on industrial quality inspection provided in Example 2 of the present invention, the determination unit includes: a setting subunit, used to set the status value of the error information according to a preset rule to obtain a target status value, wherein the status value is used to indicate the error type of the error information; and a processing unit, used to obtain operating status information based on resource occupancy data and the target status value.

[0091] Optionally, in the data processing device based on industrial quality inspection provided in Example 2 of the present invention, the processing unit also includes: a reading subunit, which is used to read the address information of the file to be synchronized in the task queue based on the file sharing protocol after receiving the file synchronization task issued by the target server, and obtain the target address information, wherein the file synchronization task includes: a synchronization task that synchronizes the industrial quality inspection data in the target machine to the storage server, and the task queue is used to store the address information of the file to be synchronized to the storage server in the target machine; the processing subunit is used to read the file to be synchronized based on the address information, and write the read file to be synchronized to the storage server.

[0092] Optionally, in the data processing device based on industrial quality inspection provided in Example 2 of the present invention, the reading sub-unit includes: an acquisition module, used to obtain the number of target threads associated with the target machine, wherein the types of target threads include: lightweight threads; a reading module, used to concurrently read the address information of the files to be synchronized in the task queue based on the number of target threads to obtain the target address information.

[0093] Optionally, in the data processing device based on industrial quality inspection provided in Embodiment 2 of the present invention, the processing sub-unit includes: a processing module, which is used to transcode and compress the images in the file to be synchronized to obtain the processed file to be synchronized when there are images in the file to be synchronized; and a processing module, which is used to establish index information of the processed file to be synchronized, and write the index information and the processed file to be synchronized to the storage server.

[0094] Optionally, in the data processing device based on industrial quality inspection provided in Embodiment 2 of the present invention, the target server includes: a server built based on a container scheduling platform, the target server includes: multiple groups of containers, the multiple groups of containers are used to execute the step of synchronizing the industrial quality inspection data in the target machine to the storage server, and the processing unit also includes: a startup unit, which is used to start a new group of containers through the target server to replace the containers in the abnormal state after receiving the file synchronization task issued by the target server, if any group of containers in the target server is in an abnormal state.

[0095] The above-mentioned data processing device based on industrial quality inspection may also include a processor and a memory. The above-mentioned processing unit 31, determination unit 32, judgment unit 33 and generation unit 34 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0096] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and by adjusting the kernel parameters, in the process of synchronizing the industrial quality inspection data in the target machine to the storage server, based on the resource occupancy of the target machine and the error information returned by the storage server, the abnormal situation of the machine is promptly warned, avoiding the situation in the related technology that the operating status of the machine cannot be monitored from multiple dimensions in a timely manner, thereby achieving the technical effect of improving the efficiency of warning of abnormal machine.

[0097] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0098] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned data processing methods based on industrial quality inspection by executing the executable instructions.

[0099] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned data processing methods based on industrial quality inspection.

[0100] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention, such as Figure 4 As shown, an embodiment of the present invention provides an electronic device 40, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, any one of the above-mentioned data processing methods based on industrial quality inspection is implemented.

[0101] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0102] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0104] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0107] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method based on industrial quality inspection, characterized in that: include: In the process of synchronizing the industrial quality inspection data in the target machine to the storage server, the resource usage data of the target machine is collected, and the error information returned by the storage server is obtained, wherein the error type of the error information includes at least one of the following: the target machine is denied access to the storage server, the connection between the target machine and the storage server fails, and the target machine is offline; Determining the operating status information of the target machine based on the resource occupancy data and the error information; Determine whether the running status information meets a preset condition and obtain a determination result; When the judgment result indicates that the operation status information satisfies the preset condition, an alarm message is generated, wherein the alarm message is used to indicate that an abnormality exists in the process of synchronizing the industrial quality inspection data in the target machine to the storage server.

2. The data processing method according to claim 1, characterized in that: Collect resource usage data of the target machine, including: Determine a target monitoring tool, wherein the target monitoring tool includes: a tool based on a cross-platform library and a command line tool, wherein the cross-platform library is used to collect the running process and resource utilization data in the target machine; The target monitoring tool is used to collect data of the target machine to obtain the resource occupancy data.

3. The data processing method according to claim 1, characterized in that: Determining the operating status information of the target machine based on the resource occupancy data and the error information includes: According to a preset rule, a state value of the error information is set to obtain a target state value, wherein the state value is used to indicate the error type of the error information; The operating status information is obtained based on the resource occupancy data and the target status value.

4. The data processing method according to claim 1, characterized in that: The steps of synchronizing the industrial quality inspection data in the target machine to the storage server include: After receiving the file synchronization task issued by the target server, based on the file sharing protocol, the address information of the file to be synchronized in the task queue is read to obtain the target address information, wherein the file synchronization task includes: a synchronization task of synchronizing the industrial quality inspection data in the target machine to the storage server, and the task queue is used to store the address information of the file to be synchronized to the storage server in the target machine; Based on the address information, the file to be synchronized is read, and the read file to be synchronized is written into the storage server.

5. The data processing method according to claim 4, characterized in that: Based on the file sharing protocol, read the address information of the file to be synchronized in the task queue to obtain the target address information, including: Acquire the number of target threads associated with the target machine, wherein the types of the target threads include: lightweight threads; Based on the number of the target threads, the address information of the files to be synchronized in the task queue is concurrently read to obtain the target address information.

6. The data processing method according to claim 4, characterized in that: Writing the read file to be synchronized into the storage server includes: When there are pictures in the file to be synchronized, transcoding and compressing the pictures in the file to be synchronized to obtain a processed file to be synchronized; The index information of the processed files to be synchronized is established, and the index information and the processed files to be synchronized are written into the storage server.

7. The data processing method according to claim 4, characterized in that: The target server includes: a server built on a container scheduling platform, the target server includes: multiple groups of containers, the multiple groups of containers are used to execute the step of synchronizing the industrial quality inspection data in the target machine to the storage server, After receiving the file synchronization task sent by the target server, the method further includes: when any group of the containers in the target server is in an abnormal state, starting a new group of containers through the target server to replace the containers in the abnormal state.

8. A data processing device based on industrial quality inspection, characterized in that: include: A processing unit, configured to collect resource occupancy data of the target machine and obtain error information returned by the storage server during synchronization of industrial quality inspection data in the target machine to the storage server, wherein the error type of the error information includes at least one of the following: denying the target machine access to the storage server, failure of the target machine to connect to the storage server, and disconnection of the target machine; A determination unit, configured to determine the operation status information of the target machine based on the resource occupancy data and the error information; A judgment unit, used to judge whether the operation status information meets a preset condition and obtain a judgment result; A generating unit is used to generate an alarm message when the judgment result indicates that the operating status information satisfies the preset condition, wherein the alarm message is used to indicate that there is an abnormality in the process of synchronizing the industrial quality inspection data in the target machine to the storage server.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the data processing method based on industrial quality inspection according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method based on industrial quality inspection as described in any one of claims 1 to 7.

Images