Anomaly detection method for manual sampling behavior

Abstract

The invention provides an anomaly detection method for a manual sampling behavior. The anomaly detection method comprises the following steps: acquiring a monitoring image sequence of a manual sampling inspection area within a specified time interval; based on a pre-trained target detection model, determining position information of a sampling association object in each monitoring image of the monitoring image sequence; based on the position information of the sampling associated object in each monitoring image, determining a sampler movement track and a sample movement track of the manual sampling inspection area within a specified time interval; based on the sampler moving track and the sample moving track, determining sampling behavior associated information of the manual sampling inspection area in a specified time interval; and based on the sampling behavior association information, determining whether the manual sampling behavior displayed by the monitoring image sequence belongs to an abnormal behavior. According to the technical scheme, the sampling behavior is evaluated in combination with target detection in deep learning, and the evaluation accuracy of sampling behavior supervision work can be effectively improved.

Description

[Technical field]

[0001] The present application relates to the field of printing manufacturing technology, and in particular to an abnormality detection method for manual sampling behavior. [Background technology]

[0002] In the printing production process, the sampling behavior of operators is of great significance to product quality and production efficiency. However, the traditional way of supervising sampling behavior is often manual, requiring quality inspectors to view the video playback of sampling behavior, observe and record the sampling behavior of operators, and then make subjective judgments. This method is not only time-consuming and labor-intensive, but also inefficient. Due to the uneven qualification level of quality inspectors, it will also cause problems such as missed inspections and wrong inspections, and it is impossible to timely discover and correct improper sampling behavior of operators.

[0003] Therefore, how to effectively supervise the sampling behavior in the printing production process has become a technical problem that needs to be solved urgently. [Summary of the invention]

[0004] The embodiment of the present application provides an anomaly detection method for manual sampling behavior, aiming to solve the technical problem in the related art that the manual supervision method of sampling behavior in the printing production process is inefficient and has a high error rate.

[0005] In a first aspect, an embodiment of the present application provides an anomaly detection method for manual sampling behavior, comprising:

[0006] Obtaining a surveillance image sequence of the manually sampled inspection area within a specified time interval;

[0007] Based on the pre-trained target detection model, determining the position information of the sampling-related objects in each surveillance image of the surveillance image sequence, wherein the sampling-related objects include: a sampler, a sample, a sample transport device, and a sample inspection station;

[0008] Determine the sampler movement trajectory and the sample movement trajectory of the manual sampling inspection area within the specified time interval based on the position information of the sampling associated object in each monitoring image;

[0009] Based on the sampler's movement trajectory and the sample's movement trajectory, determining sampling behavior association information of the manual sampling inspection area within the specified time interval, wherein the sampling behavior association information is used to reflect the possibility of abnormality in the manual sampling behavior displayed by the monitoring image sequence;

[0010] Based on the sampling behavior association information, it is determined whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior.

[0011] In one embodiment of the present application, optionally, determining the position information of the sampled associated object in each surveillance image of the surveillance image sequence based on the pre-trained target detection model includes:

[0012] Based on the pre-trained target detection model, identifying the sampled associated objects in each surveillance image in the surveillance image sequence;

[0013] The pixel position or coordinate position of the sampling associated object in each monitoring image of the monitoring image sequence is determined as the position information of the sampling associated object.

[0014] In one embodiment of the present application, optionally, before determining the position information of the sampled associated object in each surveillance image of the surveillance image sequence based on the pre-trained target detection model, the method further includes:

[0015] Acquire a sample image and a test image of a sample sampling inspection area, wherein the sample sampling inspection area includes the artificial sampling inspection area, and imaging performance characteristics of the sampling-related objects in the sample image are the same as imaging performance characteristics of the sampling-related objects in the artificial sampling inspection area;

[0016] Based on the sample image, the initial target detection model is iteratively trained until the matching degree between the target detection result of the target detection model for the test image and the actual sampled associated object in the test image is greater than or equal to a specified matching degree threshold.

[0017] In one embodiment of the present application, optionally, determining the sampler movement trajectory and the sample movement trajectory of the manual sampling inspection area within the specified time interval based on the location information of the sampling-related object in each monitoring image includes:

[0018] sequentially connecting the positions of the samplers with the same sampler identification in the monitoring image sequence according to the image arrangement order of the monitoring image sequence to obtain the sampler movement trajectory of the sampler in the manual sampling inspection area within the specified time interval; and

[0019] According to the image arrangement order of the monitoring image sequence, the positions of samples with the same sample identification in the monitoring image sequence are sequentially connected to obtain the sample movement trajectory of the sample in the manual sampling inspection area within the specified time interval.

[0020] In one embodiment of the present application, optionally, determining the sampling behavior association information of the manual sampling inspection area within the specified time interval based on the sampler movement trajectory and the sample movement trajectory includes:

[0021] Determine the relative position change information of the sampler's movement trajectory and the sample's movement trajectory with respect to the sample transport device and the sample inspection station within the specified time interval; and

[0022] Determining the spatiotemporal intersection information of the sampler's movement trajectory and the sample's movement trajectory;

[0023] Based on the relative position change information and the time-space intersection information, the sampling position, sampling times, sampling position, sampling times, single sampling duration and total sampling duration of the manual sampling inspection area within the specified time interval are determined.

[0024] In one embodiment of the present application, optionally, determining whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior based on the sampling behavior association information includes:

[0025] Encode the sampling position, sampling times, sampling position, sampling times, single sampling duration and total sampling duration of the manual sampling inspection area within the specified time interval to obtain a first feature value sequence;

[0026] Normalizing the first eigenvalue sequence to obtain a second eigenvalue sequence;

[0027] Performing weighted averaging processing on the second eigenvalues ​​in the second eigenvalue sequence to obtain the abnormality of the artificial sampling behavior displayed by the monitoring image sequence, wherein the weight of the second eigenvalue is proportional to the inverse difference level between the sampling behavior association information corresponding to the second eigenvalue and the preset safety range;

[0028] If the abnormality is within the preset abnormal range, it is determined that the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior; otherwise, it is determined that the artificial sampling behavior displayed by the monitoring image sequence is not an abnormal behavior.

[0029] In one embodiment of the present application, optionally, determining whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior based on the sampling behavior association information includes:

[0030] If the sampling behavior association information of the specified number of the sampling position, the sampling number, the sampling position, the sampling number, the single sampling duration and the total sampling duration exceeds its own preset safety range, it is determined that the manual sampling behavior displayed by the monitoring image sequence is an abnormal behavior; or

[0031] If at least one of the sampling behavior related information including the sampling position, the sampling number, the sampling position, the sampling number, the single sampling time and the total sampling time exceeds its own preset safety range, it is determined that the artificial sampling behavior displayed in the monitoring image sequence is an abnormal behavior.

[0032] In one embodiment of the present application, optionally, the method further includes:

[0033] After determining that the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior, early warning information for the abnormal behavior is generated, wherein the early warning information includes the behavior type, occurrence time and occurrence location of the abnormal behavior.

[0034] In a second aspect, an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method described in the first aspect above.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method described in the first aspect above.

[0036] The above technical solution aims at the technical problem that the manual supervision method of sampling behavior in the printing production process in the related technology is inefficient and has a high error rate. The sampler, sample, sample transport equipment and sample inspection table in the monitoring image sequence of the sampling behavior and the movement trajectory of the sampler and sample are identified through target detection in deep learning, and the various sampling behavior related information that affects the manual sampling behavior is estimated based on this information. Finally, it is determined whether the manual sampling behavior displayed in the monitoring image sequence is an abnormal behavior through the performance of these sampling behavior related information. In this way, the manual supervision method is replaced, and the monitoring of sampling behavior can be automated, which improves the efficiency of sampling behavior supervision. At the same time, the sampling behavior is evaluated in combination with target detection in deep learning. Compared with the manual supervision method, the evaluation accuracy of the sampling behavior supervision work can be effectively improved, and the smooth progress of the sampling behavior supervision work can escort the printing quality.

Brief Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A flow chart of an abnormality detection method for manual sampling behavior according to an embodiment of the present application is shown;

[0039] Figure 2 An overall schematic diagram of an anomaly detection system for manual sampling behavior according to an embodiment of the present application is shown;

[0040] Figure 3 A block diagram of a computer device according to an embodiment of the present application is shown;

[0041] Figure 4 A block diagram of a computer device according to another embodiment of the present application is shown. [Specific implementation method]

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Figure 1 A flow chart of an anomaly detection method for manual sampling behavior according to an embodiment of the present application is shown.

[0044] like Figure 1 As shown, according to an embodiment of the present application, a method for detecting anomalies in manual sampling behavior includes:

[0045] Step 102: Acquire a monitoring image sequence of the manually sampled inspection area within a specified time interval.

[0046] During the printing production process, operators (i.e., samplers) are required to conduct individual sampling inspections on printed products in a predetermined manual sampling inspection area. In order to supervise this sampling behavior, an image capture device may be set up in the manual sampling inspection area. Monitoring video in the manual sampling inspection area may be captured by the image capture device, and a monitoring image sequence within a specified time interval may be extracted from the monitoring video as a basis for evaluating the sampling behavior of the sampler within the specified time interval.

[0047] Optionally, a surveillance image is extracted from the surveillance video at intervals of a first time length and added to the surveillance image sequence.

[0048] Optionally, a plurality of third time periods are divided in the surveillance video, each of which is spaced apart by the second time period, and a specified number of surveillance images are randomly selected in each third time period and added to the surveillance image sequence.

[0049] Of course, any numerical values ​​involved in the context such as the specified time interval, the first duration, the second duration, and the third duration can be set based on actual sampling supervision requirements.

[0050] Step 104 : determining the position information of the sampled associated object in each surveillance image of the surveillance image sequence based on the pre-trained target detection model.

[0051] Next, target detection is performed on each surveillance image in the surveillance image sequence through the pre-trained target detection model, that is, the sampling-related objects in each surveillance image are identified, wherein the sampling-related objects include but are not limited to any entities related to the sampling behavior, such as the sampler, the sample, the sample transport equipment, and the sample inspection station. After the sampling-related objects in the surveillance image are identified, the position information of the sampling-related objects in the surveillance image is located, wherein the position information of the sampling-related objects in the surveillance image reflects the position change of the sampling-related objects during the sampling behavior.

[0052] In one possible design, step 104 includes: based on a pre-trained target detection model, identifying the sampled associated objects in each monitoring image in the monitoring image sequence, and determining the pixel position or coordinate position of the sampled associated objects in each monitoring image of the monitoring image sequence as the position information of the sampled associated objects.

[0053] That is, a unified coordinate system can be set for each monitoring image, and the coordinate position of the sampled associated object in the coordinate system can be determined as the position information of the sampled associated object. Alternatively, the pixel position of the sampled associated object in the monitoring image can be directly used as the position information of the sampled associated object.

[0054] Optionally, the pixel position is a set of multiple pixels imaging the sampled associated object.

[0055] Optionally, the pixel position is the position of a cluster point of a plurality of pixels of the sampled associated object imaging.

[0056] In addition, in a possible design, before step 104, the initial target detection model needs to be trained. Specifically, a sample image and a test image of the sample sampling inspection area are obtained, and based on the sample image, the initial target detection model is iteratively trained to adjust its model parameters until the target detection result of the test image by the target detection model and the actual sampling associated object in the test image have a matching degree greater than or equal to a specified matching degree threshold. The specified matching degree threshold refers to the minimum matching degree when the target detection result and the actual sampling associated object are identified as the same entity. Therefore, if the target detection result of the test image and the actual sampling associated object in the test image have a matching degree greater than or equal to the specified matching degree threshold, it means that the current target detection model can effectively detect the entity in the monitoring image, and the training can be stopped.

[0057] In a possible design, the sample sampling inspection area does not include the manual sampling inspection area, and is an area with a similar purpose to the manual sampling inspection area.

[0058] In another possible design, the sample sampling inspection area includes the artificial sampling inspection area, and the imaging performance characteristics of the sampling associated object in the sample image are the same as the imaging performance characteristics of the sampling associated object in the artificial sampling inspection area. In other words, the target detection model can be adjusted based on each artificial sampling behavior supervision task, and the sample image of the artificial sampling inspection area of ​​the artificial sampling behavior supervision task itself can be used to perform deep learning training on the target detection model.

[0059] Step 106: Determine the sampler's movement trajectory and the sample's movement trajectory in the manual sampling inspection area within the specified time interval based on the location information of the sampling-related object in each monitoring image.

[0060] A monitoring image sequence is a group of monitoring images arranged in chronological order. After obtaining the position information of the sampling-related objects of each monitoring image, the positions of the samplers and samples in the manual sampling inspection area at different time points corresponding to the monitoring image sequence can be determined, and these positions are linked according to the time distribution order of the monitoring image sequence, that is, the movement trajectory of the samplers and the movement trajectory of the samples in the manual sampling inspection area within the specified time interval are obtained.

[0061] In one possible design, step 106 includes: sequentially connecting the positions of samplers with the same sampler identification in the monitoring image sequence according to the image arrangement order of the monitoring image sequence to obtain the sampler movement trajectory of the sampler in the manual sampling inspection area within the specified time interval; and sequentially connecting the positions of samples with the same sample identification in the monitoring image sequence according to the image arrangement order of the monitoring image sequence to obtain the sample movement trajectory of the sample in the manual sampling inspection area within the specified time interval.

[0062] Specifically, there may be multiple samplers in the manual sampling inspection area. In order to distinguish the movement trajectories of different samplers, different samplers can be distinguished based on sampler identification, and the positions of samplers with the same sampler identification in the monitoring image sequence are sequentially connected to obtain their corresponding sampler movement trajectories.

[0063] Optionally, the sampler's logo is a picture logo or a character logo on the sampler's clothing.

[0064] Optionally, the sampler identification is a facial recognition image of the sampler.

[0065] Similarly, the number of samples in the manual sampling inspection area may be multiple, for example, there are multiple samplers in the manual sampling inspection area, each sampler detects one or more samples within a specified time interval, and another example is that there is one sampler in the manual sampling inspection area, and the sampler detects multiple samples within a specified time interval. Therefore, in order to distinguish the movement trajectories of different samples, different samples can be distinguished based on sample identifiers, and the positions of samples with the same sample identifier in the monitoring image sequence are sequentially connected to obtain their corresponding sample movement trajectories.

[0066] Step 108: Determine sampling behavior association information of the manual sampling inspection area within the specified time interval based on the sampler's movement trajectory and the sample's movement trajectory.

[0067] Step 110: Determine whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior based on the sampling behavior association information.

[0068] Among them, the sampling behavior association information is used to reflect the possibility of abnormality in the artificial sampling behavior displayed in the monitoring image sequence. Therefore, before further determining whether the artificial sampling behavior is abnormal, it is necessary to obtain the sampling behavior association information involved in the artificial sampling behavior. If the possibility of abnormality reflected by the sampling behavior association information is high enough, it can be determined that the artificial sampling behavior displayed in the monitoring image sequence is abnormal behavior.

[0069] Specifically, in step 108, first, the relative position change information of the sampler's movement trajectory and the sample's movement trajectory with respect to the sample transport device and the sample inspection station within the specified time interval can be determined, as well as the spatiotemporal intersection information of the sampler's movement trajectory and the sample's movement trajectory can be determined.

[0070] The relative position change information reflects the position change between the sampler and the sample and the sample transport device and the sample inspection station during the specified time interval, and the time-space intersection information reflects the overlapped track and the time period corresponding to the overlapped track during the sampler and the sample transport device and the sample inspection station during the specified time interval. Among them, the position change between the sampler and the sample transport device and the sample inspection station reflects the number of sampling. During this period, the overlapped track between the sampler and the sample and the sample transport device and the sample inspection station is the number of times the sampler looks at the sample. The starting position and the end position of this overlapped track are the sampling position and the sample viewing position, respectively. The length of time that the sampler and the sample stay at the end position of each overlapped track and the length of time corresponding to the overlapped track is the single sample viewing time, and the sum of the single sample viewing time of all overlapped tracks is the total sample viewing time. On this basis, the sampling position, sampling times, sample viewing position, sample viewing times, single sample viewing time and total sample viewing time of the manual sampling inspection area within the specified time interval can be determined based on the relative position change information and the time-space intersection information. Of course, the sampling behavior related information includes but is not limited to the sampling position, sampling times, sampling location, sampling times, single sampling duration and total sampling duration of the manual sampling inspection area within the specified time interval, and may also include any other related information related to the sampling supervision task requirements.

[0071] In step 110, first, the sampling position, sampling times, sampling positions, sampling times, single sampling duration, and total sampling duration of the manual sampling inspection area within the specified time interval are encoded to obtain a first feature value sequence, and then the first feature value sequence is normalized to obtain a second feature value sequence. That is, each sampling behavior association information is encoded and standardized so that the feature values ​​corresponding to each sampling behavior association information are at the same level, which is convenient for subsequent calculations.

[0072] Next, a weighted average process is performed on the second eigenvalues ​​in the second eigenvalue sequence to obtain the abnormality of the artificial sampling behavior displayed by the monitoring image sequence.

[0073] Among them, the weight of the second eigenvalue is proportional to the reverse difference level between the sampling behavior association information corresponding to the second eigenvalue and the preset safety range. In other words, the more the sampling behavior association information deviates from the preset safety range corresponding to itself, the higher the weight of the corresponding second eigenvalue is, and the weight reflects the abnormal degree of the sampling behavior association information to which the second eigenvalue belongs. Therefore, the abnormality obtained by weighted averaging the second eigenvalues ​​in the second eigenvalue sequence reflects the possibility of abnormality of the artificial sampling behavior displayed in the monitoring image sequence under the combined influence of multiple sampling behavior association information.

[0074] At this point, if the abnormality is within the preset abnormal range, it is determined that the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior, otherwise, it is determined that the artificial sampling behavior displayed by the monitoring image sequence is not an abnormal behavior.

[0075] The above technical solution uses target detection in deep learning to identify the sampler, sample, sample transport equipment and sample inspection table in the monitoring image sequence of the sampling behavior, as well as the movement trajectory of the sampler and the sample, and based on this information, estimates the various sampling behavior association information that affects the manual sampling behavior, and finally determines whether the manual sampling behavior displayed in the monitoring image sequence is an abnormal behavior through the performance of these sampling behavior association information. In this way, instead of replacing the manual supervision method, the monitoring of the sampling behavior can be automated, which improves the efficiency of the sampling behavior supervision work. At the same time, the sampling behavior is evaluated in combination with target detection in deep learning. Compared with the manual supervision method, it can effectively improve the evaluation accuracy of the sampling behavior supervision work, and the smooth progress of the sampling behavior supervision work can escort the printing quality.

[0076] In addition, after determining that the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior, early warning information for the abnormal behavior is generated to prompt the abnormal artificial sampling behavior in time, so that the sampler can correct his own behavior in time. Among them, the early warning information includes the behavior type, occurrence time and occurrence location of the abnormal behavior. The behavior type reflects the specific direction of the abnormality of the sampler's artificial sampling behavior, such as too low sampling frequency, too long sampling time, etc.

[0077] On the basis of the above embodiments, the amount and / or frequency of abnormal behaviors in a long period of time (such as one month) can also be obtained. If the amount and / or frequency of abnormal behaviors are higher than the preset safety threshold, the acquisition of the scrap rate in the long period of time can be triggered. If the scrap rate is lower than the preset safety rate, it means that the quality of the sampling behavior in the long period of time is low, and the sampling results cannot truly reflect the actual quality level of the printed products. In this way, the overall sampling behavior can be standardized to improve the production quality of printed products.

[0078] Figure 2 An overall schematic diagram of an anomaly detection system for manual sampling behavior according to an embodiment of the present application is shown.

[0079] like Figure 2 As shown, the anomaly detection system for manual sampling behavior includes: a video acquisition module, a router, a video analysis module, an alarm information synthesis module, an alarm publishing module and a third-party platform.

[0080] Among them, the video acquisition module, as the main equipment for front-end data acquisition, is responsible for capturing the video images of the operator's sampling behavior, and collecting video data of each printing working area through the collector corresponding to the multi-channel video stream of the video device.

[0081] The router is responsible for data transmission, transmitting the video data collected by the video acquisition module to the video analysis module safely and efficiently.

[0082] The video analysis module pre-processes and intelligently analyzes the transmitted video data, detects whether the sampling behavior of the operator meets the process specification requirements, that is, whether it is an abnormal behavior, and sends the image and analysis results to the alarm information synthesis module. Specifically, the video analysis module is equipped with video analysis algorithms and business-related algorithms. The video analysis algorithm has target detection and target tracking functions, and the business-related algorithm has off-duty detection and sampling detection functions.

[0083] The alarm information synthesis module is equipped with an image encoder, a video encoder, a video memory, an alarm synthesizer, etc. It transmits images, analysis results, and alarm information to the front-end user interface through a multi-channel video stream via a streaming media server. At the same time, the alarm information synthesis module sends the alarm data to the publisher of the alarm publishing module, which is then synchronously transmitted to the front-end user interface and the third-party platform for alarm. The third-party platform can store, display, and manage the alarm data of abnormal behavior. In this way, alarms can be issued through multiple channels to maximize the alarm information being known to management personnel.

[0084] In addition, the front-end user interface, video analysis module, and video acquisition module are all configured as management objects of the back-end management module and are coordinated by the back-end management module.

[0085] Thus, by real-time monitoring and analysis of operators' sampling behaviors, the system can quickly capture any abnormal or non-compliant operations, thus greatly improving the efficiency and accuracy of supervising sampling behaviors in printing work. Compared with traditional manual supervision methods, this intelligent supervision system based on deep learning can continuously and comprehensively cover the sampling behaviors of all operators, reduce omissions and blind spots, and ensure the comprehensiveness and fairness of supervision.

[0086] At the same time, the system can timely detect and correct improper behavior of operators, thus effectively reducing production risks and safety hazards. In printing and other manufacturing industries, the sampling behavior of operators has an important impact on product quality and production efficiency. Through the application of intelligent supervision systems, enterprises can better control the production process and ensure product quality and production safety.

[0087] In addition, by introducing advanced artificial intelligence technology, companies can better respond to market changes and customer needs, and improve their overall operational level and competitiveness.

[0088] Furthermore, this system can also serve as an important tool for employee training and quality improvement. By recording and analyzing the sampling behavior data of operators, the system enables enterprises to understand the skill level and operation habits of employees, making it easier to formulate more targeted training plans. This data-based training method can more accurately identify the training needs of employees, improve training results, and enhance the overall quality and skill level of employees.

[0089] In addition, in one embodiment, the present application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the method described in any of the above embodiments can be implemented.

[0090] In one embodiment, the present application further provides a computer device, which may be a client, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the method described in any of the above embodiments can be implemented.

[0091] Any of the above-mentioned computer devices in the embodiments of the present application may exist in various forms, including but not limited to:

[0092] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0093] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0094] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys, wearable devices, and portable car navigation devices.

[0095] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0096] (5) Other electronic devices with data interaction functions.

[0097] In addition, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to perform the following steps:

[0098] Obtaining a surveillance image sequence of the manually sampled inspection area within a specified time interval;

[0099] Based on the pre-trained target detection model, determining the position information of the sampling-related objects in each surveillance image of the surveillance image sequence, wherein the sampling-related objects include: a sampler, a sample, a sample transport device, and a sample inspection station;

[0100] Determine the sampler movement trajectory and the sample movement trajectory of the manual sampling inspection area within the specified time interval based on the position information of the sampling associated object in each monitoring image;

[0101] Based on the sampler's movement trajectory and the sample's movement trajectory, determining sampling behavior association information of the manual sampling inspection area within the specified time interval, wherein the sampling behavior association information is used to reflect the possibility of abnormality in the manual sampling behavior displayed by the monitoring image sequence;

[0102] Based on the sampling behavior association information, it is determined whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior.

[0103] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant description in the aforementioned method embodiment. In order to avoid repetition, they will not be described one by one here.

[0104] The technical solution of the present application is described in detail above in conjunction with the accompanying drawings. The technical solution of the present application identifies the sampler, sample, sample transport equipment and sample inspection table in the monitoring image sequence of the sampling behavior and the movement trajectory of the sampler and sample through target detection in deep learning, and estimates the various sampling behavior association information that affects the manual sampling behavior based on this information, and finally determines whether the manual sampling behavior displayed in the monitoring image sequence is an abnormal behavior through the performance of these sampling behavior association information. In this way, instead of the manual supervision method, the monitoring of the sampling behavior can be automated, and the efficiency of the sampling behavior supervision work is improved. At the same time, the sampling behavior is evaluated in combination with the target detection in deep learning. Compared with the manual supervision method, the evaluation accuracy of the sampling behavior supervision work can be effectively improved, and the printing quality is protected through the smooth progress of the sampling behavior supervision work.

[0105] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0106] It should be understood that, although the terms first, second, etc. may be used to describe the eigenvalue sequences in the embodiments of the present application, these eigenvalue sequences should not be limited to these terms. These terms are only used to distinguish the eigenvalue sequences from each other. For example, without departing from the scope of the embodiments of the present application, the first eigenvalue sequence may also be referred to as the second eigenvalue sequence, and similarly, the second eigenvalue sequence may also be referred to as the first eigenvalue sequence.

[0107] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0108] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0112] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting anomalies in artificial sampling behavior, characterized in that: include: Obtaining a surveillance image sequence of the manually sampled inspection area within a specified time interval; Based on the pre-trained target detection model, determining the position information of the sampling-related objects in each surveillance image of the surveillance image sequence, wherein the sampling-related objects include: a sampler, a sample, a sample transport device, and a sample inspection station; Determine the sampler movement trajectory and the sample movement trajectory of the manual sampling inspection area within the specified time interval based on the position information of the sampling associated object in each monitoring image; Based on the sampler's movement trajectory and the sample's movement trajectory, determining sampling behavior association information of the manual sampling inspection area within the specified time interval, wherein the sampling behavior association information is used to reflect the possibility of abnormality in the manual sampling behavior displayed by the monitoring image sequence; Based on the sampling behavior association information, it is determined whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior.

2. The method according to claim 1, characterized in that The determining, based on the pre-trained target detection model, the location information of the sampled associated object in each surveillance image of the surveillance image sequence comprises: Based on the pre-trained target detection model, identifying the sampled associated objects in each surveillance image in the surveillance image sequence; The pixel position or coordinate position of the sampling associated object in each monitoring image of the monitoring image sequence is determined as the position information of the sampling associated object.

3. The method according to claim 2, characterized in that Before determining the position information of the sampled associated object in each surveillance image of the surveillance image sequence based on the pre-trained target detection model, the method further includes: Acquire a sample image and a test image of a sample sampling inspection area, wherein the sample sampling inspection area includes the artificial sampling inspection area, and imaging performance characteristics of the sampling-related objects in the sample image are the same as imaging performance characteristics of the sampling-related objects in the artificial sampling inspection area; Based on the sample image, the initial target detection model is iteratively trained until the matching degree between the target detection result of the target detection model for the test image and the actual sampled associated object in the test image is greater than or equal to a specified matching degree threshold.

4. The method according to claim 3, characterized in that The step of determining the sampler movement trajectory and the sample movement trajectory of the manual sampling inspection area within the specified time interval based on the location information of the sampling associated object in each monitoring image includes: sequentially connecting the positions of the samplers with the same sampler identification in the monitoring image sequence according to the image arrangement order of the monitoring image sequence to obtain the sampler movement trajectory of the sampler in the manual sampling inspection area within the specified time interval; and According to the image arrangement order of the monitoring image sequence, the positions of samples with the same sample identification in the monitoring image sequence are sequentially connected to obtain the sample movement trajectory of the sample in the manual sampling inspection area within the specified time interval.

5. The method according to any one of claims 1 to 4, characterized in that The determining, based on the sampler's movement trajectory and the sample's movement trajectory, sampling behavior association information of the manual sampling inspection area within the specified time interval includes: Determine the relative position change information of the sampler's movement trajectory and the sample's movement trajectory with respect to the sample transport device and the sample inspection station within the specified time interval; and Determining the spatiotemporal intersection information of the sampler's movement trajectory and the sample's movement trajectory; Based on the relative position change information and the time-space intersection information, the sampling position, sampling times, sampling position, sampling times, single sampling duration and total sampling duration of the manual sampling inspection area within the specified time interval are determined.

6. The method according to claim 5, characterized in that The determining, based on the sampling behavior association information, whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior includes: Encode the sampling position, sampling times, sampling position, sampling times, single sampling duration and total sampling duration of the manual sampling inspection area within the specified time interval to obtain a first feature value sequence; Normalizing the first eigenvalue sequence to obtain a second eigenvalue sequence; Performing weighted averaging processing on the second eigenvalues ​​in the second eigenvalue sequence to obtain the abnormality of the artificial sampling behavior displayed by the monitoring image sequence, wherein the weight of the second eigenvalue is proportional to the inverse difference level between the sampling behavior association information corresponding to the second eigenvalue and the preset safety range; If the abnormality is within the preset abnormal range, it is determined that the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior; otherwise, it is determined that the artificial sampling behavior displayed by the monitoring image sequence is not an abnormal behavior.

7. The method according to claim 5, characterized in that The determining, based on the sampling behavior association information, whether the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior includes: If the sampling behavior association information of the specified number of the sampling position, the sampling number, the sampling position, the sampling number, the single sampling duration and the total sampling duration exceeds its own preset safety range, it is determined that the manual sampling behavior displayed by the monitoring image sequence is an abnormal behavior; or If at least one of the sampling behavior related information including the sampling position, the sampling number, the sampling position, the sampling number, the single sampling time and the total sampling time exceeds its own preset safety range, it is determined that the artificial sampling behavior displayed in the monitoring image sequence is an abnormal behavior.

8. The method according to claim 1, characterized in that Also includes: After determining that the artificial sampling behavior displayed by the monitoring image sequence is an abnormal behavior, early warning information for the abnormal behavior is generated, wherein the early warning information includes the behavior type, occurrence time and occurrence location of the abnormal behavior.

9. A computer device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are configured to enable the processor to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are configured to execute the method according to any one of claims 1 to 8.

Images