Data processing method, device, server and storage medium

Abstract

The disclosed embodiment discloses a data processing method, device, server and storage medium, the method comprising: receiving the first defect detection results of each first user on the associated image of each first user through the target client, and updating the first defect detection results of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache the set type data and cache the query results; in response to the query request of the second user, the query result is determined based on the updated data cache space of the first middleware. Compared with directly querying in the database, transferring the data to the cache space and querying in the cache space has higher query efficiency and strong real-time performance; the first defect detection results are added to the cache space to improve the accuracy of data query.

Description

Technical Field

[0001] The present disclosure relates to the field of database technology, and in particular to a data processing method, device, server and storage medium. Background Art

[0002] In common data analysis systems, in order to extract database production history records, for example, common ETL (Extract-Transform-Load) tools are used to build an ETL server, query the database regularly, and transfer the data to other databases according to certain rules. The back-end program accesses the data, extracts the data, and returns it to the front-end for display.

[0003] However, the timed extraction of traditional ETL tools cannot meet the real-time requirements. If the timed extraction task is adjusted more frequently, it will greatly increase the access pressure of the database and reduce the timeliness of the query. In addition, the source of data production records is single, resulting in low query accuracy. Summary of the invention

[0004] The embodiments of the present disclosure provide a data processing method, device, server and storage medium to improve the timeliness and accuracy of data query.

[0005] In a first aspect, an embodiment of the present disclosure provides a data processing method, including:

[0006] Receive the first defect detection result of each first user's associated image through the target client, and update the first defect detection result of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache the set type data and cache the query results;

[0007] In response to the query request of the second user, a query result is determined based on the updated data cache space of the first middleware.

[0008] In a second aspect, an embodiment of the present disclosure provides a data processing device, including:

[0009] A data updating unit, configured to receive the first defect detection result of each first user on the associated image of each first user through the target client, and update the first defect detection result of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache the set type data and cache the query result;

[0010] A query unit is used to determine a query result based on the updated data cache space of the first middleware in response to a query request of the second user.

[0011] In a third aspect, an embodiment of the present disclosure provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0012] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of any of the above methods when executed by a processor.

[0013] The disclosed embodiment has the following beneficial effects: the target server receives the first defect detection result of each associated image of the first user; the target server updates the first defect detection result of each associated image to the cache space of the first middleware (used to cache data of a set type and cache query results), so that when there is a query request, the corresponding query request is responded to, and the query result is determined based on the updated cache space of the first middleware. In this way, on the one hand, querying the result in the cache space of the first middleware reduces the data access pressure of the database and improves the timeliness of data query; on the other hand, the first defect detection result is added to the cache space of the first middleware, which improves the accuracy of data query. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic diagram of an application scenario of a data processing method provided by an embodiment of the present disclosure;

[0016] Figure 2 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0017] Figure 3 A flowchart of the interaction between Canal and MySQL provided in one embodiment of the present disclosure;

[0018] Figure 4 A schematic diagram of an image judgment interface provided by an embodiment of the present disclosure;

[0019] Figure 5 A schematic diagram of a performance comparison of common cache components provided by an embodiment of the present disclosure;

[0020] Figure 6A flow chart of interaction between a Canal Client and a Canal Server provided in one embodiment of the present disclosure;

[0021] Figure 7 A schematic diagram of a query process provided by an embodiment of the present disclosure;

[0022] Figure 8 A sample diagram of a product defect trend provided by an embodiment of the present disclosure;

[0023] Fig. 9 A schematic diagram of the structure of a data processing device provided by an embodiment of the present disclosure;

[0024] Fig.10 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure more clear, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure.

[0026] For ease of understanding, the terms involved in the embodiments of the present disclosure are explained below:

[0027] (1) TMS (Task Management System): used to process, analyze and display the yield data of glass substrates.

[0028] (2)MySQL: A relational database stores data in different tables, which increases speed and flexibility.

[0029] (3) Redis: open source distributed cache middleware.

[0030] (4) Canal: Open source MySQL log subscription middleware.

[0031] (5) ETL: Data warehouse technology, used to describe the process of extracting, transforming, and loading data from the source to the destination. Kettle is an open source ETL tool.

[0032] (6) MES (Manufacturing Execution System) is a production information management system for the workshop execution level of manufacturing enterprises.

[0033] (7) DFS (Distributed file system): A network server component that can query and manage data on the network.

[0034] (8) ADC (Automatic Defects Classification) platform, also known as automated detection platform, can integrate defect recognition models, such as defect recognition models based on neural networks. In this way, artificial intelligence, image recognition and other technologies can be used to automatically detect and classify key defects in the panel production process, which can greatly improve the company's quality inspection efficiency, accuracy, yield and production efficiency.

[0035] Next, the defect detection process and principle of the neural network-based defect recognition model are explained.

[0036] The neural network-based defect recognition model can be integrated in an automated detection device, which can be one or more devices capable of performing artificial intelligence reasoning (visual recognition) and training tasks using GPU computing resources. The reasoning task based on artificial intelligence can be a defect recognition task based on a defect recognition model, which is mainly a neural network model based on deep learning. For example, the defect recognition model can be based on a feedforward neural network. The feedforward network can be implemented as an acyclic graph, in which nodes are arranged in layers. Typically, the feedforward network topology includes an input layer and an output layer, which are separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for generating an output in the output layer. The network nodes are fully connected to the nodes in the adjacent layers via edges, but there are no edges between the nodes in each layer. The data received at the nodes of the input layer of the feedforward network is propagated (i.e., "feedforward") to the nodes of the output layer via an activation function, and the activation function calculates the state of the nodes of each continuous layer in the network based on coefficients ("weights"), and the coefficients are respectively associated with each of the edges connecting these layers. The output of the defect recognition model can take various forms, and the present disclosure is not limited to this. The defect recognition model may also include other neural network models, such as a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a generative adversarial network (GAN) model, and a transformation attention (TransFormer) model, but is not limited thereto. Other neural network models known to those skilled in the art may also be used.

[0037] Neural network models usually need to be obtained through training. In the disclosed embodiment, the automated detection device also trains the neural network model, which mainly includes the following steps: selecting a network topology; using a set of training data representing the problem modeled by the network; and adjusting the weights until the network model exhibits the minimum error for all instances of the training data set. For example, during a supervised learning training process for a neural network, the output generated by the network in response to an input representing an instance in the training data set is compared with the "correct" labeled output of the instance; an error signal representing the difference between the output and the labeled output is calculated; and when the error signal is propagated backward through the layers of the network, the weights associated with the connection are adjusted to minimize the error. When the error of each output generated from an instance of the training data set is minimized, the neural network model is considered to be "trained" and can be used for artificial intelligence reasoning tasks.

[0038] The reasoning and training process of the neural network model involves a large number of parameters, activation values, and gradient value buffers, each of which must be completely updated in each training iteration, which places very high demands on computing and throughput capabilities. GPUs have strong parallel capabilities and high video memory bandwidth, and are extremely efficient in processing calculations associated with training deep neural networks. Therefore, using a GPU cluster integrated with multiple GPUs can effectively improve the training and reasoning speed of the neural network model, and the product defect recognition cluster 202 can also schedule and allocate hardware resources.

[0039] In at least one embodiment of the present disclosure, the automated inspection equipment may be configured to identify product defect content in a product image through a defect recognition model based on a first defect recognition task, wherein the product defect content includes any one or more of the product defect type, defect location, and defect size. The product defect content includes any one or more of the product defect type, defect location, and defect size. After receiving the first defect recognition task sent by the TMS, the automated inspection equipment identifies the product defect content of the product image stored in the product image database through the trained defect recognition model, that is, identifies any one or more of the product defect type, defect location, and defect size.

[0040] In an embodiment of the present disclosure, based on the first defect recognition task, the implementation method of identifying the product defect content in the product image through the defect recognition model is as follows: first, the product image is scaled to a fixed pixel size MxN (or not scaled), and then the MxN image is sent to a deep convolutional neural network (VGG / Resnet / MobileNet, etc.); through multiple layers of convolutional layers, activation layers, and pooling layers, the feature map of the entire image is obtained; the feature map is input into the screening region network (ZF / SSD / RPN, etc.), and after calculation, a proposal region is obtained; then, convolution and pooling operations are performed on the proposal region to obtain a regional feature map (proposal feature) of the proposal region, and then the regional feature map (proposal feature) is sent to the subsequent full connection and softmax network for classification (classification, that is, what kind of defect the proposal is), and the defect category with the highest probability is obtained as the final classification result, and the category and probability are recorded. In addition, the coordinates and size of the proposal region represent the location and size of the defect. The method for identifying product defect content based on the defect model may adopt similar variations of the above method or other methods known to those skilled in the art, and the present disclosure does not limit this.

[0041] For example, in the semiconductor industry, defect types may include particles, remain, line defects, holes, splashes, wrinkles, film color, bubbles, etc., and more subcategories may be made based on each defect type. For example, particle types may be divided into normal particles P1, sand particles P2, and wall particles P3 based on particle defects of different forms. The defect location may indicate the specific coordinate location of the defect in the product image, thereby determining the specific coordinate location of the defect on the circuit board, the relationship between the defect and the background circuit (e.g., whether the defect is completely within the gate, intersecting with it, or near the gate), etc. The defect size may include the length of the defect or the area of ​​the defect.

[0042] In addition, the process of how the image acquisition device AOI acquires images and transmits images is explained.

[0043] The product images generated during the product manufacturing process may include product images acquired by the image acquisition device AOI during the manufacturing process, and may also include product images acquired through other means and stored in the product image database. For example, after the AOI device acquires the product image, it can be directly stored in the product image database, or it can be stored in the factory database, and then the product image in the factory database is transferred to the product image database. Transmitting the product image to the product image database from the factory database can ensure that the distributed product defect analysis system will not affect other data in the factory when calling, storing, and other operations on the product image in the product image database. The product image database can be a distributed file system (Distributed File System, DFS) or a relational database (Oracle, Redis, and MySQL).

[0044] For example, when the AOI device inspects the screen products on the production line, the AOI device can send a single product manufacturing message as a product manufacturing message to the MES system after the inspection of a screen (or a large glass substrate screen, also known as Glass) is completed, and send a picture file (.jpg / .gls) to the product image database, or it can also send a single product manufacturing message to all activated devices in the current factory. The AOI device may also send batch product manufacturing messages in units of LOT (1 Lot contains 20 Glasses, each Glass is a single large glass substrate screen). For example, when a LOT inspection is completed, the AOI will send a batch product manufacturing message (for example, batch product information) as a product manufacturing message to the MES system or any other related equipment. In the product manufacturing process, batch products (LOT products) are usually processed and produced as units, that is, the product manufacturing sites, process flows, etc. experienced by batch products are all the same. Therefore, batch product manufacturing messages are usually transmitted as a product manufacturing message in the factory, which can more efficiently manage product manufacturing messages.

[0045] For example, when parsing a batch product manufacturing message, TMS can obtain the number of product images contained in multiple products, the product type corresponding to the product image, the size of the product image, etc. The product image can be an image taken in any step of the semiconductor screen from the substrate to the deposition, etching and finally the box. When generating a product defect analysis request message, TMS needs to determine the product type to be analyzed in the product defect analysis task and the type of product defect analysis task to be performed (for example, artificial intelligence recognition, manual recognition, etc.). Afterwards, TMS can also perform message format verification on the product defect analysis request message, and after determining that the format of the product defect analysis request message is qualified, distribute the product defect analysis task to other devices (such as product defect recognition cluster, client device, etc.) by sending the product defect analysis request message. The product defect analysis task type is used to indicate different types of product defect analysis tasks. The product defect analysis task includes a first defect recognition task, wherein the first defect recognition task refers to a task of defect recognition based on artificial intelligence, which can be performed by an automated detection platform. Therefore, TMS can generate a first defect recognition task based on the product manufacturing message and send the first defect recognition task to the automated detection platform.

[0046] In the actual practice, as the TMS system is put into use, many bad records will be generated when the product goes through the TMS detection program. In order to reflect the value of data, how to extract and integrate historical data to meet real-time requirements and improve user query efficiency is an urgent problem to be solved.

[0047] In the related art, the ADC platform is used to detect the image of the glass substrate, and a large amount of product defect information is obtained, such as the layer (film information), site, product model, Glass (glass substrate) number and Panel (panel) number, Lot (batch) number, defect code, maintenance and other information of this inspection. However, these basic data are stored in the database. When the user has a query request, on the one hand, there are cases where the ADC judgment is wrong or cannot be recognized. On the other hand, the query in the server increases the query pressure and reduces the timeliness of data query.

[0048] In order to screen, extract and transfer these basic data, and integrate the display function into the data analysis module of the TMS system, it is helpful for users to intuitively observe or discover the incidence and repair rate of various product defects, and then optimize the production process, thereby improving product quality and production yield.

[0049] To this end, the disclosed embodiment proposes a data extraction method based on Canal subscription to MySQL logs, which can transfer data to a database, as well as other middleware, such as Redis data cache space or MQ message middleware, etc., so that real-time and incremental processing of data changes can be achieved, and the processing results can be stored in the database or written into the cache middleware according to business needs, so that when the back-end program queries the data, real-time and efficient response can be achieved.

[0050] The present disclosure provides a data processing method, in which a target server receives the first defect detection results of each first user (such as a reviewer) on the associated image of each first user through a target client, and updates the first defect detection results of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache set type data and cache query results; in response to a query request from a second user (such as a queryer), the query result is determined based on the updated data cache space of the first middleware. In this way, the query is queried in the data cache space of the first middleware instead of in the entire database, which improves the timeliness of the query; and the data cache space also integrates the manual first defect detection results, which improves the accuracy of the query.

[0051] After introducing the design concept of the embodiment of the present disclosure, the following briefly introduces the application scenarios to which the technical solution of the embodiment of the present disclosure can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiment of the present disclosure and are not limited. In specific implementation, the technical solution provided by the embodiment of the present disclosure can be flexibly applied according to actual needs.

[0052] refer to Figure 1 , which is a schematic diagram of an application scenario of a data processing method provided by an embodiment of the present disclosure, wherein each glass substrate includes multiple detection points. For example, through an image acquisition device, each detection point corresponds to an image, and the images of the same batch (for example, 20 pieces) of glass substrates are input into the ADC platform to obtain the detection results. Then, according to the results of manual re-judgment received from the target client, the detection results are updated to the Redis middleware, and subsequent queries can be performed in Redis, and the final query results are displayed on the target client.

[0053] Of course, the method provided in the embodiment of the present disclosure is not limited to Figure 1 The application scenarios shown in the figure can also be used in other possible application scenarios, and the embodiments of the present disclosure are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.

[0054] To further illustrate the technical solution provided by the embodiments of the present disclosure, the following is a detailed description of the technical solution in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present disclosure provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present disclosure.

[0055] Combine the following Figure 1 The application scenario shown illustrates the technical solution provided by the embodiment of the present disclosure.

[0056] refer to Figure 2 The present disclosure provides a data processing method, which is applied to a TMS server and includes the following steps:

[0057] S201, receiving the first defect detection results of each first user's associated image through the target client, and updating the first defect detection results of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache set type data and cache query results.

[0058] S202: In response to a query request of the second user, determine a query result based on the updated data cache space of the first middleware.

[0059] In the disclosed embodiment, the target server receives the first defect detection result of each associated image of the first user, and the first defect detection result is determined by the target client; the target server updates the first defect detection result of each associated image to the cache space of the first middleware (used to cache data of a set type and cache query results), so that when there is a query request, it responds to the corresponding query request and determines the query result based on the updated cache space of the first middleware. In this way, on the one hand, querying the result in the cache space of the first middleware reduces the data access pressure of the database and improves the timeliness of data query; on the other hand, the first defect detection result is added to the cache space of the first middleware, which improves the accuracy of data query.

[0060] First, the process of obtaining the second defect detection result is described in combination with the application scenario of the embodiment of the present disclosure:

[0061] In actual application, in order to improve the accuracy of detection, before inputting the batch detection data into the pre-trained defect recognition model, the batch detection data is analyzed and verified to determine the integrity of the batch detection data.

[0062] After determining that the batch detection data is complete, for any set of batch detection data, the batch detection data is input into a pre-trained defect recognition model to apply the defect recognition model to parse the batch detection data to obtain a corresponding second defect detection result; the second defect detection result is received, and the second defect detection result is sent to a target client through a second middleware. In a specific example, the defect recognition model is a defect recognition model based on a neural network.

[0063] The batch detection data may be detection data determined according to a preset number of target images, and the target images may be images obtained after photographing the detection points of each glass substrate. In actual application, a certain number of glass substrates of the same type may be detected together, and the detection data obtained in this way is batch detection data. In this way, for any set of batch detection data, it is input into a pre-trained defect recognition model, and the defect recognition model obtains the corresponding second defect detection result by parsing the batch detection data.

[0064] Among them, the message can be pushed to the defect recognition model through the MQ message middleware, and the message includes the bad information in the batch detection data, the path of the image, etc. The defect recognition model will call the python script of the GPU (Graphics Processing Unit) Server, use the corresponding model according to the site to parse the bad image, and give the inference result of the bad type, that is, the second defect detection result.

[0065] In a specific example, the second defect detection result is basic data, such as the Layer (film information), site, product model, Glass (glass substrate) number and Panel (panel) number, Lot (batch) number, defective Code, maintenance and other information of the defective point.

[0066] The TMS server receives the second defect detection result from the defect recognition model, and sends the second defect detection result to the TMS client through the second middleware (such as MQ message middleware).

[0067] Secondly, the target server still takes the TMS server as an example, and the target client still takes the TMS client as an example, and the method of obtaining batch detection data is explained:

[0068] For example, the method of obtaining batch detection data is related to the current mode (online mode and offline mode) of the TMS server, as follows:

[0069] In the first case, if the current mode is the online mode of the TMS server, a message instruction from the MES system is received to grab a preset number of target images from the DFS system.

[0070] The TMS server receives a message instruction from the MES system in the online mode, and fetches a preset number of target images from the DFS system to complete the Track in process. Exemplarily, the target images may be in the form of Lot data.

[0071] In the second case, if the current mode is the offline mode of the TMS server, the mounted batch file is monitored to see if there is any change. If so, a preset number of target images are captured.

[0072] Among them, in offline mode, the TMS server will monitor the changes of the mounted batch files, such as data increase, that is, determine whether a new batch file (Lot data) is generated. If so, it will capture a preset number of target images and then decide to execute the Track process.

[0073] Parse a preset number of target images to obtain batch detection data.

[0074] Among them, grabbing Lot data will parse Lot and Glass information, and store batch inspection data in the database.

[0075] In actual application, a preset number of target images can be parsed by the first server, and then the batch detection data can be sent to the first client through the first server, so that the first client can store the batch detection data in the first middleware, the second middleware and the local cache component according to the type of the batch detection data.

[0076] The first server is, for example, a Canal Server, and the first client is, for example, a Canal Client. A specific implementation method may be that the Canal Server disguises itself as a MySQL Slave node and sends a dump request to the master database (MySQL database) to obtain the binlog log, so as to obtain the Lot data corresponding to a preset number of target images. After being parsed by the Canal Server, it is processed by the Canal Client, and the processing result can be stored in the MySQL database, the first middleware (Redis middleware), the second middleware (MQ message middleware), and the local cache component.

[0077] In a specific example, Figure 3A flow chart of the interaction between Canal and MySQL is shown, wherein the first server is Canal Server, the first client is Canal Client, the database is MySQL database, the first middleware is Redis middleware, and the second middleware is MQ message middleware. In this way, Canal Server pulls logs from the MySQL database, then parses the logs and hands them over to Canal Client to store them in the MySQL database, Redis middleware, and MQ message middleware respectively.

[0078] As mentioned above, the TMS server sends the second defect detection result obtained through the defect recognition model to the target client. Specifically, the TMS server can send the second defect detection result to each TMS client through the MQ message middleware, and hand it over to the engineer or operator for review. The second defect detection result includes the basic information of each defective point, such as panel information, coordinate information, etc., and also includes the defective information of ADC reasoning. These defective information will be persisted in the database for subsequent analysis and query.

[0079] In S201, the first user is, for example, a re-judge. The re-judge re-judges the second defect detection result through the TMS client. In the actual application process, since there are many second defect detection results, in order to improve efficiency, each re-judge can see the images that need to be re-judged assigned to him by the TMS service after logging into his own TMS client, which are called associated images. The TMS client sends the first defect detection result to the TMS server, and updates the first defect detection result of each associated image to the first middleware.

[0080] Specifically, when the user logs in to the TMS client and opens the image judgment interface, the task issued by the MTS Server will be automatically pulled. Each client can process a piece of Glass at the same time. For defective points for which the ADC has no inference results, engineers or operators will manually judge and obtain the first defect detection result (image judgment result). The image judgment result of each defective point will also be recorded in the database.

[0081] In a specific example, Figure 4 A schematic diagram of an image judgment interface is shown, wherein area 41 is an enlarged interface of an image of a defective point, and area 42 is a first defect detection result interface. In addition, the number of defective points is only used for illustration and does not constitute a specific limitation.

[0082] After each glass is judged by the TMS client, it will be reported to the TMS Server. When all glasses of a batch have been judged, the TMS Server in online mode will send a Track out message to the MES to inform the MES that the data of this batch has been processed and can be transferred to the next process. If it is in offline mode, only the processing information will be recorded.

[0083] In a specific example, the first middleware is a Redis middleware, and the data cache space of the middleware is used to cache data of a set type and cache query results. For queries with user-defined screening conditions, such as product models and defective types, the query results are stored in the Redis cache so that subsequent queries do not need to access the database.

[0084] In addition, in order to prevent excessive memory usage, a preset expiration time is set for the cached data in the data cache space of the Redis middleware so that the cached data in the Redis middleware can be updated in real time.

[0085] Exemplarily, the set type of data includes hot data in the benchmark data and at least one type of data in the benchmark data. Among them, the benchmark data includes product ID, bad code, site type, etc. Since this type of data is often used as a screening condition, it can also be stored in Redis to avoid the need to query the database every time a query is made. It can be found directly in Redis. Hot data is data that users frequently access, such as data for the day, data for the month, and the comparative frequency of user clicks on the display. In order to ensure the smoothness of the system page, this type of data can be stored in Redis, which can improve query efficiency and enhance user experience.

[0086] In S202, in response to a query request from a second user (such as a query person), a query result is determined based on the updated data cache space of the first middleware (Redis middleware).

[0087] In the actual application process, in order to make full use of resources, save network bandwidth, and reduce the occupancy of Redis services, a local cache component can be added. In this way, multi-level caching can be combined with MySQL database and Redis middleware. When data needs to be queried, the local cache component is queried first. If the query hits the cache component, it is returned directly. Otherwise, the Redis middleware is queried. If the Redis middleware does not have the data, the MySQL database is queried. When the data is queried, the queried data is added to the multi-level cache.

[0088] Specifically, during the query process, since the local cache query speed is fast, the query results are obtained in the local cache component first. If the query results cannot be obtained in the local cache component, the query results can be determined in the updated Redis data cache space. In addition, there is a situation where the query results cannot be determined in the updated Redis data cache space. At this time, the query results can be determined in the target bad history table (stored in the MySQL database), where the situation that cannot be queried is, for example, that the corresponding query space does not store the corresponding data to be queried. In order to improve the efficiency of subsequent queries, each query result can be stored in the Redis data cache space to increase the probability of being able to query in the Redis data cache space next time, without having to query in the MySQL database. In a specific example, the query condition is, for example, the number of glass substrates whose number of bad points accounts for more than 60% of the number of detection points of the entire glass substrate in the past week.

[0089] As mentioned above, in the order of local cache components, Redis middleware and MySQL database queries, the query method of multi-level cache middleware is used to improve data query efficiency. On the basis of ensuring data accuracy, the user experience in the software analysis module (such as the report page used to display query results) is greatly optimized.

[0090] For a specific example, see Figure 5 , shows a schematic diagram of performance comparison of common cache components, wherein the common cache components shown include type 1-Caffeine, type 2-ConcurrentLinkedHashMap, type 3-LinkedHashMap, type 4-Guava, type 5-Ehcache2_Lru, type 6-Ehcache3_Lru or type 7-Infinispan_Old_Lru, etc. Figure 4 The horizontal axis is the common cache component types, and the vertical axis is the data throughput, with the unit being ops / s, the number of operations per second.

[0091] In actual application, caching hot data in Redis can improve query efficiency. However, when updating data, the cache needs to be adjusted. Otherwise, the query may return old data, resulting in inaccurate data displayed to users.

[0092] In the disclosed embodiment, the data update is monitored by the Canal Server to update Redis. Redis can be updated in the Canal Client data processing logic, or the cache can be updated in the backend after the message is pushed to the MQ message middleware. For the local cache component, it can also be updated with the help of the message queue. For data that is not sensitive to real-time, a shorter expiration time can also be set to make it invalid automatically. If the data changes frequently and is frequently queried, try to use a shorter expiration time.

[0093] In addition, with the local cache component of Spring Boot, the query results can be automatically loaded into the local cache. It should be noted that different invalidation conditions should be set according to different types of cache. If it is a basic configuration that does not change for a long time, such as site information, product information, etc., it can be loaded into the cache when the system starts, and then invalidated when these configurations are updated.

[0094] During the query process, when the corresponding data cannot be found in the local cache and Redis middleware, the target bad history table in the database is queried. Exemplarily, the target bad history table is determined in the following way:

[0095] Based on the first defect detection results of each associated image and the basic defect history table, a target defect history table is determined; wherein, at least one set of second defect detection results is stored in the basic defect history table; the second defect detection results are obtained by applying a preset defect recognition model to batch detection data, and the batch detection data is the detection data corresponding to a preset number of target images.

[0096] Among them, each group of batch inspection data corresponds to a group of second defect detection results, each group of second defect detection results is stored in the basic defect history table, and then the target defect table is determined based on the first defect detection results of each associated image and the basic defect history table.

[0097] Method 1: Use the first defect detection result of each associated image to update the basic defect history table and determine the target defect history table.

[0098] For example, the basic defect table includes a column of data for the second defect detection result, and the first defect detection result is added to the basic defect table as a column of data to form a target defect table. The first defect detection result and the second defect detection result may be the same or different.

[0099] Method 2: Based on the basic defect table, according to the preset update conditions, based on the first defect detection results of each associated image, a target defect history table is generated, wherein the preset update conditions are determined according to the data update time or data storage type.

[0100] For example, based on the defect history table, a new target defect history table is generated, and the target defect history table stores data determined according to preset update conditions, wherein the preset update conditions are determined according to the data update time and / or the data storage type. In this way, the target defect history table can include summary tables by day and month, and detailed tables by lot, glass, and panel. In this way, querying in the target defect history table according to the refined summary table can reduce the amount of data to be queried and improve query efficiency.

[0101] For example, Canal Client is developed in pure Java, which is convenient for integration with the backend Spring Boot program. It only requires the introduction of SDK (Software Development Kit) and configuration of relevant information of Canal Server. When Canal Server receives data changes from the MySQL database, it will send the changes to CanalClient for processing. In this way, by monitoring the bad history table, when new data is inserted, it is updated to the summary table by day and month, and the detailed table by lot, glass, and panel. Updates to frequently used hot data, such as the change trend of the yield of the day, the operation trend of the equipment and lot, etc., can be stored in Redis for quick query by the client. If the existing data in Redis is updated, it should be invalidated to prevent dirty data from being found. In a specific example, Figure 6 A flow chart of interaction between a Canal Client and a Canal Server is shown, wherein the first client is a Canal Client and the first server is a Canal Server.

[0102] In the above embodiment, after the Canal environment is built, the Canal Server subscribes to the history table of the TMS database, and the Canal Client monitors data changes, sorts out the change information, and stores it in the summary table and the detail table. Frequently used benchmark information and hot spot data are stored in the Redis cache, and the cache is updated when the data changes to ensure the consistency of the cache and the database. When the user consults through the front-end page, the back-end program can quickly query the data and return it to the front-end for display, which greatly improves the user experience.

[0103] In addition, after the user judges the image (re-judgment), if he wants to view the changing trends of various defective rates on that day, he can open the data analysis module of the TMS client, set the filtering conditions and click query, which will initiate a query request.

[0104] In a specific example, Figure 7A query process is shown, wherein when the local cache cannot find the result, the query is then performed in Redis and MySQL.

[0105] The TMS client receives the query results and displays the report. For example, use the echarts component to draw a chart. Or display the response results in other forms. For a specific example, see Figure 8 , shows a sample graph of product defect trends, where the horizontal axis is the batch code and the vertical axis is the percentage of defect type 1 and defect type 2 respectively; in addition, the batch code is only used for illustration, and the actual batch code can be set according to product-related information.

[0106] In summary, in the embodiments of the present disclosure, if the user has a need to query a large amount of data, and the cache cannot be used because the amount of data is too large, or the conditions for data query change frequently, when querying with the help of the MySQL database, a professional search engine component, such as Elasticsearch, can be introduced. For the timeliness of the local cache, if the Redis version used is above 6.0, you can use Redis's new feature - client cache. This feature can update the local cache in time when the Redis content changes, ensure the consistency of the local cache and the Redis cache, significantly reduce network bandwidth usage, and do not rely on the message middleware, thereby reducing the complexity of the architecture.

[0107] In addition, Canal's yield data extraction solution uses less resources and puts less pressure on the server than traditional solutions, and the data is real-time. In terms of data storage, a multi-level cache is added between the database layer and the backend program, and a reliable solution is used to ensure the consistency of the cached data. Directly querying data from the cache can obtain query results faster, improve user experience, and reduce the pressure on the MySQL database.

[0108] It should be noted that in the above example, the first defect detection result is determined based on the second defect detection result, that is, the second defect detection result is automatically identified by the application of the defect detection model, and the first defect detection result is the re-judgment result based on this. In actual application, the two may not be related, such as two different types of defect detection or detection of different parts of the glass substrate, etc. This is just an example and does not constitute a specific limitation.

[0109] like Fig. 9 As shown, based on the same inventive concept as the above-mentioned data processing method, the embodiment of the present disclosure further provides a data processing device, including a data updating unit 901 and a query unit 902 .

[0110] The data updating unit 901 is used to receive the first defect detection result of each first user's associated image through the target client, and update the first defect detection result of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache the set type data and cache the query results;

[0111] The query unit 902 is configured to determine a query result based on the updated data cache space of the first middleware in response to the query request of the second user.

[0112] In some optional implementations, a bad history table determination unit is further included, which is configured to, after receiving the first defect detection result of each first user on the associated image of each first user through the target client:

[0113] Based on the first defect detection results of each associated image and the basic defect history table, a target defect history table is determined; wherein, at least one set of second defect detection results is stored in the basic defect history table; the second defect detection results are obtained by applying a preset defect detection model to batch detection data, and the batch detection data is the detection data corresponding to a preset number of target images.

[0114] In some optional implementations, the query unit 902 is further configured to:

[0115] If the query result cannot be determined in the updated first middleware data cache space, the query result is determined in the target bad history table.

[0116] In some optional implementations, the bad history table determining unit is further configured to:

[0117] Apply the first defect detection results of each associated image to update the basic defect history table to determine the target defect history table; or

[0118] On the basis of the basic defect table, according to the preset update conditions, based on the first defect detection results of each associated image, a target defect history table is generated;

[0119] The preset update condition is determined according to the data update time and / or the data storage type.

[0120] In some optional implementations, the query unit 902 is further configured to:

[0121] If the query result cannot be obtained in the local cache component, the query result is determined in the updated first middleware data cache space.

[0122] In some optional implementations, a pre-processing unit is further included, which is used for receiving the first defect detection result of each first user on the associated image of each user through the target client before the method further includes:

[0123] For any set of batch test data, the batch test data is input into a pre-trained defect recognition model, and the defect recognition model is applied to analyze the batch test data to obtain the corresponding second defect detection result; wherein the defect recognition model is a defect recognition model based on a neural network

[0124] A second defect detection result is received, and the second defect detection result is sent to a target client through a second middleware.

[0125] In some optional embodiments, an integrity determination unit is further included, which is used to: before sending the batch inspection data to the defect recognition model:

[0126] Determine the integrity of batch inspection data.

[0127] In some optional embodiments, the method further includes a data determination unit, which is used to determine the batch detection data in the following manner:

[0128] If the current mode is the online mode of the target server, a message instruction from the MES system is received to grab a preset number of target images from the DFS system;

[0129] If the current mode is the offline mode of the target server, monitor whether the mounted batch files have changed. If so, grab the preset number of target images;

[0130] Parse a preset number of target images to obtain batch detection data.

[0131] In some optional implementations, the data determination unit is specifically configured to:

[0132] A preset number of target images are analyzed by the first server to obtain batch detection data.

[0133] In some optional implementations, a cache unit is further included, which is used to: after parsing a preset number of target images to obtain batch detection data:

[0134] The batch detection data is sent to the first client through the first server, so that the first client stores the batch detection data correspondingly to the first middleware, the second middleware and the local cache component according to the type of the batch detection data.

[0135] In some optional implementations, the first middleware is Redis middleware, and the method further includes:

[0136] For the cached data in the data cache space of the Redis middleware, a preset expiration time is set so that the cached data in the Redis middleware is updated in real time.

[0137] In some optional implementations, the set type data includes hot spot data in the benchmark data and at least one type of data in the benchmark data.

[0138] The data processing device provided in the embodiment of the present disclosure adopts the same inventive concept as the above-mentioned data processing method and can achieve the same beneficial effects, which will not be described in detail here.

[0139] Based on the same inventive concept as the above data processing method, the embodiment of the present disclosure also provides an electronic device, which can be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), a server, etc. Fig.10 As shown, the electronic device may include a processor 1001 and a memory 1002 .

[0140] Processor 1001 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software units in the processor.

[0141] The memory 1002 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and units. The memory may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1002 in the embodiment of the present disclosure can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0142] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; the above computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM, Random Access Memory), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)) and other media that can store program codes.

[0143] Alternatively, if the above-mentioned integrated unit of the present disclosure is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present disclosure. The aforementioned storage medium includes: a mobile storage device, a random access memory (RAM, Random Access Memory), a magnetic storage (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage (such as CD, DVD, BD, HVD, etc.), and a semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), a solid state drive (SSD)) and other various media that can store program codes.

[0144] The above embodiments are only used to introduce the technical solutions of the present invention in detail, but the description of the above embodiments is only used to help understand the methods of the embodiments of the present invention and should not be understood as limiting the embodiments of the present invention. Any changes or substitutions that can be easily thought of by technicians in this technical field should be included in the protection scope of the embodiments of the present invention.

Claims

1. A data processing method, wherein: include: Receive the first defect detection result of each first user's associated image through the target client, and update the first defect detection result of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache the set type data and cache the query results; In response to a query request of a second user, determining a query result based on the updated data cache space of the first middleware; After receiving the first defect detection result of each first user on the associated image of each first user through the target client, the method further includes: Based on the first defect detection results of each associated image and the basic defect history table, a target defect history table is determined; wherein, at least one set of second defect detection results is stored in the basic defect history table; the second defect detection result is obtained by applying a preset defect recognition model to batch detection data, and the batch detection data is detection data corresponding to a preset number of target images.

2. The method according to claim 1, wherein: The method further comprises: If the query result cannot be determined in the updated first middleware data cache space, the query result is determined in the target bad history table.

3. The method according to claim 1, wherein: The step of determining a target defect history table based on the first defect detection results of each associated image and the basic defect history table includes: Apply the first defect detection results of each associated image to update the basic defect history table to determine a target defect history table; or On the basis of the basic defect history table, generating a target defect history table according to a preset update condition and based on the first defect detection result of each associated image; The preset update condition is determined according to the data update time and / or the data storage type.

4. The method according to claim 1, wherein: The determining the query result based on the updated data cache space of the first middleware includes: If the query result cannot be obtained in the local cache component, the query result is determined in the updated first middleware data cache space.

5. The method according to claim 1, wherein: Before receiving the first defect detection result of each first user on the associated image of each user through the target client, the method further includes: For any set of batch detection data, the batch detection data is input into a pre-trained defect recognition model, so as to apply the defect recognition model to parse the batch detection data and obtain a corresponding second defect detection result; wherein the defect recognition model is a defect recognition model based on a neural network; The second defect detection result is received, and the second defect detection result is sent to a target client through a second middleware.

6. The method according to claim 5, wherein: Before inputting the batch inspection data into the pre-trained defect recognition model, the method further includes: The integrity of the batch test data is determined.

7. The method according to claim 5, wherein: The batch detection data is obtained in the following manner: If the current mode is the online mode of the target server, a message instruction from the MES system is received to grab a preset number of target images from the DFS system; If the current mode is the offline mode of the target server, monitor whether the mounted batch file has changed, and if so, capture the preset number of target images; The preset number of target images are analyzed to obtain the batch detection data.

8. The method according to claim 7, wherein: The step of analyzing the preset number of target images to obtain the batch detection data includes: The preset number of target images are analyzed by the first server to obtain the batch detection data.

9. The method according to claim 7, wherein: After parsing the preset number of target images to obtain the batch detection data, the method further includes: The batch detection data is sent to the first client through the first server, so that the first client stores the batch detection data correspondingly to the first middleware, the second middleware and the local cache component according to the type of the batch detection data.

10. The method according to claim 1, wherein: The first middleware is Redis middleware, and the method further includes: For the cached data in the data cache space of the Redis middleware, a preset expiration time is set so that the cached data in the Redis middleware is updated in real time.

11. The method according to any one of claims 1 to 10, wherein: The set type data includes hot spot data in the benchmark data and at least one type of data in the benchmark data.

12. A data processing device, wherein: Applied to the data processing method according to claim 1, the data processing device comprises: A data updating unit, configured to receive the first defect detection result of each first user on the associated image of each first user through the target client, and update the first defect detection result of each associated image to the data cache space of the first middleware; wherein the data cache space of the first middleware is used to cache the set type data and cache the query result; A query unit is used to respond to the query request of the second user and determine the query result based on the updated data cache space of the first middleware.

13. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having computer program instructions stored thereon, wherein: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images