Video detection method, device, equipment, system, storage medium and program product
Through video detection methods, real-time detection of objects to be tested in the government and enterprise system is solved, and the problem of inefficient detection caused by multiple pictures taken in the existing technology is solved, which improves detection efficiency and adapts to different business needs.
Patent Information
- Application Number
- CN202411253052.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-15
AI Technical Summary
In the prior art, the installation and maintenance inspection efficiency of the government and enterprise systems is low, and multiple pictures are required to take pictures, resulting in low detection efficiency.
By obtaining video input, shooting videos and detecting objects to be detected in real time in the video, displaying the detection results of each object to be detected, supporting different detection types of multiple objects to be detected, avoiding multiple pictures to be taken need.
Improve inspection efficiency, reduce the workload of operators, and adapt to the flexibility of different business needs.
Smart Images

Figure CN118968388A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of communication technology, and in particular to a video detection method, apparatus, device, system, storage medium, and program product. Background Art
[0002] In many practical scenarios, the maintenance of application systems and the detection of related security are essential processes. These scenarios include but are not limited to enterprise-level applications, Internet of Things (IoT) systems, e-commerce platforms, financial transaction systems, medical information systems, and government and enterprise systems (such as digital service platforms). Taking the government and enterprise system as an example, the installation and maintenance (referred to as installation and maintenance) and daily inspections of government and enterprise services are important factors affecting business quality and customer perception. To this end, existing technologies have proposed a variety of government and enterprise service installation and maintenance detection solutions.
[0003] One of the detection schemes is that the installation and maintenance personnel use the installation and maintenance application (Application, APP) to take pictures of the objects to be detected in the order of the detection tasks, and detect the pictures to obtain the detection results. However, in this method, since the pictures need to be taken in the order of the detection tasks for detection, there may be a situation where the pictures do not meet the requirements and need to be taken multiple times. Therefore, this method may lead to a low detection efficiency of the detection scheme. Summary of the invention
[0004] Embodiments of the present application provide a video detection method, apparatus, device, system, storage medium, and program product to improve detection efficiency.
[0005] In a first aspect, an embodiment of the present application provides a video detection method, which is applied to a first electronic device, including:
[0006] Get the first input;
[0007] In response to the first input, capturing the video;
[0008] Detecting the video and displaying the detection result of each object to be detected;
[0009] The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0010] Optionally, the video is an original image captured by the first electronic device during a period from the start to the end of each shooting.
[0011] Optionally, the detection type includes one or more of the following:
[0012] Construction safety information corresponding to the object to be detected;
[0013] Service information corresponding to the object to be detected;
[0014] The device resource information corresponding to the object to be detected;
[0015] The construction process information corresponding to the object to be detected.
[0016] Optionally, during the process of shooting the video, the shooting order of the objects to be detected is random.
[0017] Optionally, the detecting the video includes:
[0018] The video is detected in real time during the shooting process.
[0019] Optionally, the display order of the detection results satisfies at least one of the following:
[0020] The display order of the detection results is consistent with the shooting order of the objects to be detected in the video;
[0021] The display order of the detection results is consistent with the order in which the objects to be detected are detected;
[0022] In the case where the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of preset detection ranges.
[0023] Optionally, the detecting of the video includes at least one of the following:
[0024] Detecting the video by the first electronic device;
[0025] The video is detected through the cloud.
[0026] Optionally, the detecting the video and displaying the detection result of each object to be detected includes:
[0027] Selecting a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected;
[0028] Performing detection based on the information of the target object to be detected, and displaying the detection result of the target object to be detected;
[0029] The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value, or the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0030] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0031] Optionally, the information of the target object to be detected includes any of the following:
[0032] The frame to be detected;
[0033] Identifications of the frame to be detected and the target object to be detected;
[0034] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0035] Optionally, the video is detected through the cloud, including:
[0036] Sending the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected;
[0037] The information to be detected is determined based on a video including at least two objects to be detected.
[0038] Optionally, the information to be detected includes the video, or information of the target object to be detected.
[0039] Optionally, the method further includes:
[0040] Sending detection task information to the cloud;
[0041] The detection task information is used to determine a target processing model, and the target processing model is used to detect the information to be detected.
[0042] Optionally, the first electronic device is a user terminal device.
[0043] In a second aspect, an embodiment of the present application further provides a video detection method, which is applied to a second electronic device, including:
[0044] Acquiring information to be detected, wherein the information to be detected is acquired based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0045] Perform detection based on the information to be detected to obtain a detection result of each object to be detected;
[0046] The detection result of each object to be detected is sent to the first electronic device.
[0047] Optionally, performing detection based on the information to be detected to obtain a detection result of each object to be detected includes:
[0048] Using the target processing model to detect the information to be detected, and obtaining the detection result of each object to be detected;
[0049] The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0050] Optionally, performing detection based on the information to be detected to obtain a detection result of each object to be detected includes:
[0051] Sending information of the target processing model and the information to be detected to a third electronic device, wherein the third electronic device is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0052] The target processing model includes a first type of model or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0053] Optionally, the method further includes:
[0054] Acquire detection task information, wherein the detection task information is used to determine the target processing model.
[0055] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0056] Optionally, the information of the target object to be detected includes any of the following:
[0057] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0058] Identifications of the frame to be detected and the target object to be detected;
[0059] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0060] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0061] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0062] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0063] Optionally, the test result satisfies at least one of the following:
[0064] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0065] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0066] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0067] The detection result includes detection results of some or all of the at least two objects to be detected.
[0068] Optionally, the second electronic device is located in the cloud.
[0069] In a third aspect, an embodiment of the present application further provides a video detection method, which is applied to a third electronic device, including:
[0070] Receiving information of a target processing model and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected shot by the first electronic device, and the at least two objects to be detected correspond to different detection types;
[0071] Using the target processing model to detect the information to be detected, and obtaining a detection result of each object to be detected;
[0072] The detection result of each object to be detected is sent to the first electronic device.
[0073] Optionally, the target processing model includes: a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than the processing capability of the first type of model.
[0074] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0075] Optionally, the information of the target object to be detected includes any of the following:
[0076] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0077] Identifications of the frame to be detected and the target object to be detected;
[0078] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0079] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0080] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1.
[0081] Optionally, the test result satisfies at least one of the following:
[0082] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0083] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0084] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0085] The detection result includes detection results of some or all of the at least two objects to be detected.
[0086] Optionally, the third electronic device is located in the cloud.
[0087] In a fourth aspect, an embodiment of the present application further provides a video detection device, applied to a first electronic device, comprising:
[0088] A first acquisition module, used for acquiring a first input;
[0089] A first processing module, configured to capture the video in response to the first input;
[0090] A second processing module is used to detect the video and display the detection result of each object to be detected;
[0091] The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0092] Optionally, the video is an original image captured by the first electronic device during a period from the start to the end of each shooting.
[0093] Optionally, the detection type includes one or more of the following:
[0094] Construction safety information corresponding to the object to be detected;
[0095] Service information corresponding to the object to be detected;
[0096] The device resource information corresponding to the object to be detected;
[0097] The construction process information corresponding to the object to be detected.
[0098] Optionally, during the process of shooting the video, the shooting order of the objects to be detected is random.
[0099] Optionally, the first processing module is further used to perform real-time detection on the video during shooting.
[0100] Optionally, the display order of the detection results satisfies at least one of the following:
[0101] The display order of the detection results is consistent with the shooting order of the objects to be detected in the video;
[0102] The display order of the detection results is consistent with the order in which the objects to be detected are detected;
[0103] In the case where the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of preset detection ranges.
[0104] Optionally, the second processing module is further used for at least one of:
[0105] Detecting the video by the first electronic device;
[0106] The video is detected through the cloud.
[0107] Optionally, the second processing module is further used to:
[0108] Selecting a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected;
[0109] Performing detection based on the information of the target object to be detected, and displaying the detection result of the target object to be detected;
[0110] The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value, or the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0111] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0112] Optionally, the information of the target object to be detected includes any of the following:
[0113] The frame to be detected;
[0114] Identifications of the frame to be detected and the target object to be detected;
[0115] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0116] Optionally, the second processing module is further used to:
[0117] Sending the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected;
[0118] The information to be detected is determined based on a video including at least two objects to be detected.
[0119] Optionally, the information to be detected includes the video, or information of the target object to be detected.
[0120] Optionally, the device may further include:
[0121] A first sending module, used for sending detection task information to the cloud;
[0122] The detection task information is used to determine a target processing model, and the target processing model is used to detect the information to be detected.
[0123] Optionally, the first electronic device is a user terminal device.
[0124] In a fifth aspect, an embodiment of the present application further provides a video detection device, which is applied to a second electronic device, including:
[0125] A first acquisition module, configured to acquire information to be detected, wherein the information to be detected is acquired based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0126] A second acquisition module is used to perform detection based on the information to be detected to obtain a detection result of each object to be detected;
[0127] The first sending module is used to send the detection result of each object to be detected to the first electronic device.
[0128] Optionally, the second acquisition module is further used to:
[0129] Using the target processing model to detect the information to be detected, and obtaining the detection result of each object to be detected;
[0130] The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0131] Optionally, the second acquisition module is further used to:
[0132] Sending information of the target processing model and the information to be detected to a third electronic device, wherein the third electronic device is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0133] The target processing model includes a first type of model or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0134] Optionally, the device may further include:
[0135] The third acquisition module is used to acquire detection task information, wherein the detection task information is used to determine the target processing model.
[0136] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0137] Optionally, the information of the target object to be detected includes any of the following:
[0138] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0139] Identifications of the frame to be detected and the target object to be detected;
[0140] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0141] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0142] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0143] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0144] Optionally, the test result satisfies at least one of the following:
[0145] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0146] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0147] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0148] The detection result includes detection results of some or all of the at least two objects to be detected.
[0149] Optionally, the second electronic device is located in the cloud.
[0150] In a sixth aspect, an embodiment of the present application further provides a video detection device, which is applied to a third electronic device, including:
[0151] A first receiving module is used to receive information of a target processing model and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected taken by the first electronic device, and the at least two objects to be detected correspond to different detection types;
[0152] A first acquisition module is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0153] The first sending module is used to send the detection result of each object to be detected to the first electronic device.
[0154] Optionally, the target processing model includes: a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than the processing capability of the first type of model.
[0155] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0156] Optionally, the information of the target object to be detected includes any of the following:
[0157] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0158] Identifications of the frame to be detected and the target object to be detected;
[0159] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0160] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0161] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1.
[0162] Optionally, the test result satisfies at least one of the following:
[0163] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0164] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0165] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0166] The detection result includes detection results of some or all of the at least two objects to be detected.
[0167] Optionally, the third electronic device is located in the cloud.
[0168] In a seventh aspect, an embodiment of the present application further provides a video detection device, applied to a first electronic device, comprising: a processor and a transceiver;
[0169] The transceiver is used to obtain a first input;
[0170] The processor is configured to shoot the video in response to the first input; detect the video and display the detection result of each object to be detected;
[0171] The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0172] Optionally, the video is an original image captured by the first electronic device during a period from the start to the end of each shooting.
[0173] Optionally, the detection type includes one or more of the following:
[0174] Construction safety information corresponding to the object to be detected;
[0175] Service information corresponding to the object to be detected;
[0176] The device resource information corresponding to the object to be detected;
[0177] The construction process information corresponding to the object to be detected.
[0178] Optionally, during the process of shooting the video, the shooting order of the objects to be detected is random.
[0179] Optionally, the processor is further used to perform real-time detection on the video during shooting.
[0180] Optionally, the display order of the detection results satisfies at least one of the following:
[0181] The display order of the detection results is consistent with the shooting order of the objects to be detected in the video;
[0182] The display order of the detection results is consistent with the order in which the objects to be detected are detected;
[0183] In the case where the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of preset detection ranges.
[0184] Optionally, the processor is further configured to perform at least one of the following:
[0185] Detecting the video by the first electronic device;
[0186] The video is detected through the cloud.
[0187] Optionally, the processor is further configured to:
[0188] Selecting a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected;
[0189] Performing detection based on the information of the target object to be detected, and displaying the detection result of the target object to be detected;
[0190] The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value, or the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0191] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0192] Optionally, the information of the target object to be detected includes any of the following:
[0193] The frame to be detected;
[0194] Identifications of the frame to be detected and the target object to be detected;
[0195] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0196] Optionally, the processor is further configured to:
[0197] Sending the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected;
[0198] The information to be detected is determined based on a video including at least two objects to be detected.
[0199] Optionally, the information to be detected includes the video, or information of the target object to be detected.
[0200] Optionally, the processor is further configured to:
[0201] Sending detection task information to the cloud;
[0202] The detection task information is used to determine a target processing model, and the target processing model is used to detect the information to be detected.
[0203] Optionally, the first electronic device is a user terminal device.
[0204] In an eighth aspect, an embodiment of the present application further provides a video detection device, applied to a second electronic device, comprising: a processor and a transceiver;
[0205] The transceiver is used to obtain information to be detected, wherein the information to be detected is obtained based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0206] The processor is used to perform detection based on the information to be detected to obtain a detection result of each object to be detected;
[0207] The transceiver is further used to send the detection result of each object to be detected to the first electronic device.
[0208] Optionally, the processor is further configured to:
[0209] Using the target processing model to detect the information to be detected, and obtaining the detection result of each object to be detected;
[0210] The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0211] Optionally, the processor is further configured to:
[0212] Sending information of the target processing model and the information to be detected to a third electronic device, wherein the third electronic device is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0213] The target processing model includes a first type of model or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0214] Optionally, the processor is further configured to:
[0215] Acquire detection task information, wherein the detection task information is used to determine the target processing model.
[0216] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0217] Optionally, the information of the target object to be detected includes any of the following:
[0218] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0219] Identifications of the frame to be detected and the target object to be detected;
[0220] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0221] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0222] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0223] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0224] Optionally, the test result satisfies at least one of the following:
[0225] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0226] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0227] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0228] The detection result includes detection results of some or all of the at least two objects to be detected.
[0229] Optionally, the second electronic device is located in the cloud.
[0230] In a ninth aspect, an embodiment of the present application further provides a video detection device, applied to a third electronic device, comprising: a processor and a transceiver;
[0231] The transceiver is used to receive information of the target processing model and information to be detected sent by the second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected taken by the first electronic device, and the at least two objects to be detected correspond to different detection types;
[0232] The processor is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0233] The transceiver is further used to send the detection result of each object to be detected to the first electronic device.
[0234] Optionally, the target processing model includes: a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than the processing capability of the first type of model.
[0235] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0236] Optionally, the information of the target object to be detected includes any of the following:
[0237] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0238] Identifications of the frame to be detected and the target object to be detected;
[0239] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0240] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0241] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1.
[0242] Optionally, the test result satisfies at least one of the following:
[0243] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0244] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0245] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0246] The detection result includes detection results of some or all of the at least two objects to be detected.
[0247] Optionally, the third electronic device is located in the cloud.
[0248] In a tenth aspect, an embodiment of the present application further provides a video detection system, including: a first electronic device and a second electronic device, wherein:
[0249] The first electronic device is used to obtain a first input; in response to the first input, shoot the video; detect the video and display the detection result of each object to be detected; wherein the video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0250] The second electronic device is used to obtain information to be detected, wherein the information to be detected is obtained based on the video shot by the first electronic device; perform detection based on the information to be detected to obtain a detection result of each object to be detected; and send the detection result of each object to be detected to the first electronic device.
[0251] In an eleventh aspect, an embodiment of the present application further provides a video detection system, including: a first electronic device, a second electronic device, and a third electronic device, wherein:
[0252] The first electronic device is used to obtain a first input; in response to the first input, shoot the video; detect the video and display the detection result of each object to be detected; wherein the video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0253] The second electronic device is used to obtain information to be detected, wherein the information to be detected is obtained based on the video shot by the first electronic device; and send information of a target processing model and the information to be detected to the third electronic device, wherein the target processing model is used to detect the information to be detected;
[0254] The third electronic device is used to receive the information of the target processing model and the information to be detected sent by the second electronic device; use the target processing model to detect the information to be detected to obtain the detection result of each object to be detected; and send the detection result of each object to be detected to the first electronic device.
[0255] In a twelfth aspect, an embodiment of the present application further provides a communication device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the steps in the video detection method described above when executing the program.
[0256] In a thirteenth aspect, an embodiment of the present application further provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps in the video detection method described above are implemented.
[0257] In a fourteenth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the video detection method as described above.
[0258] In an embodiment of the present application, the first electronic device captures a video when a first input for capturing a video is obtained, and detects the video, and displays the detection result of each object to be detected in the video. Since the detection is based on the video in the embodiment of the present application, and the video includes at least two objects to be detected and the at least two objects to be detected correspond to different detection types, the scheme of the embodiment of the present application can simultaneously perform different detection types on multiple objects to be detected in the video, thereby avoiding the problem of low detection efficiency caused by the need to capture a picture of a detection object multiple times and detect the picture in the prior art, and further, the scheme of the embodiment of the present application can improve the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0259] Figure 1 It is one of the flow charts of the video detection method provided in the embodiment of the present application;
[0260] Figure 2 This is the second flow chart of the video detection method provided in the embodiment of the present application;
[0261] Figure 3 This is the third flow chart of the video detection method provided in the embodiment of the present application;
[0262] Figure 4 This is the fourth flow chart of the video detection method provided in the embodiment of the present application;
[0263] Figure 5 This is one of the terminal interface schematic diagrams provided in the embodiment of the present application;
[0264] Figure 6 This is the second terminal interface schematic diagram provided in the embodiment of the present application;
[0265] Figure 7 This is the third terminal interface schematic diagram provided in the embodiment of the present application;
[0266] Figure 8 This is the fourth terminal interface schematic diagram provided in the embodiment of the present application;
[0267] Fig. 9 This is the fifth terminal interface schematic diagram provided in the embodiment of the present application;
[0268] Fig.10 This is the sixth terminal interface schematic diagram provided in the embodiment of the present application;
[0269] Fig.11 This is the seventh terminal interface schematic diagram provided in the embodiment of the present application;
[0270] Fig.12 This is the eighth terminal interface schematic diagram provided in the embodiment of the present application;
[0271] Fig.13 It is one of the structural diagrams of the video detection device provided in the embodiment of the present application;
[0272] Fig.14 This is the second structural diagram of the video detection device provided in the embodiment of the present application;
[0273] Fig.15 This is the third structural diagram of the video detection device provided in the embodiment of the present application;
[0274] Fig.16 This is the fourth structural diagram of the video detection device provided in the embodiment of the present application;
[0275] Fig.17 This is the fifth structural diagram of the video detection device provided in the embodiment of the present application;
[0276] Fig.18 This is the sixth structural diagram of the video detection device provided in the embodiment of the present application;
[0277] Fig.19 It is one of the structural diagrams of the video detection system provided in the embodiment of the present application;
[0278] Fig. 20 This is the second structural diagram of the video detection system provided in the embodiment of the present application. DETAILED DESCRIPTION
[0279] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0280] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.
[0281] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0282] See also Figure 1 , Figure 1 : is a flow chart of a video detection method provided in an embodiment of the present application, which is applied to a first electronic device. The first electronic device may be a user terminal device, such as a terminal, virtual reality (VR) glasses, a recorder, etc. Figure 1 As shown, the following steps are included:
[0283] Step 101: Obtain a first input.
[0284] In the embodiment of the present application, the first input may include input from the user, triggering of the first electronic device itself, and other inputs. The user input includes but is not limited to inputs such as a single click or touch of the user on the first electronic device; the triggering of the first electronic device itself includes but is not limited to the expiration of a timer set for the first electronic device to trigger video shooting, preset detection configuration information, and the like. In the embodiment of the present application, the first input can be understood as an input for shooting a video, or as an input for triggering detection.
[0285] Step 102: In response to the first input, shoot the video.
[0286] In practical applications, the first electronic device has a video shooting module such as a camera, so when the first input is obtained, the camera of the first electronic device can be used to shoot a video.
[0287] In an embodiment of the present application, the video is an original image captured by the first electronic device during the time period from the start of each shooting to the end of the shooting. Among them, the meaning of the original image can be understood as an image that has not been processed by editing, modification, etc. That is to say, the unedited image during the period from the start of each shooting to the end of the shooting is used as the video here. Therefore, in an embodiment of the present application, the video can be understood as a video obtained by shooting in a "one-shot" manner. In this way, the information of multiple objects to be detected can be obtained in real time, and since the video is shot continuously without intermediate editing or modification, the real situation of the environment in which the object to be detected is located can be retained to the greatest extent, ensuring the reliability of the subsequent detection results. At the same time, due to the reduction of links such as editing, the efficiency of video acquisition can also be improved, thereby further improving the detection efficiency and speed.
[0288] Wherein, in the process of shooting the video, the shooting order of the object to be detected is random. Wherein, "random" can be understood as there is no order requirement for shooting the object to be detected. Therefore, in the embodiment of the present application, since there is no need to fix the shooting order of the object to be detected, the object to be detected can be detected after it is detected, so that the detection of the object to be detected can be flexibly realized, and there is no need to set the detection order in the terminal APP in advance, which can meet the different detection requirements of different businesses.
[0289] The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0290] In an embodiment of the present application, the object to be detected may be, for example, the operator's clothing, the helmet worn, the work card, the equipment to be detected, the environment to be detected, and the like. The at least two objects to be detected correspond to different detection types, which can be understood as different objects to be detected corresponding to different detection types, or the same object to be detected has different detection types. Among them, the detection types may include multiple types, such as construction safety information, service information, equipment resource information, construction process information, and the like. For example, for a safety helmet, the corresponding detection type may include construction safety information, and the detection type corresponding to the operator's clothing may be service information. For another example, the detection type that the label may correspond to may include equipment resource information, construction process information, and the like. By setting different detection types, multiple types of detection may be achieved simultaneously and in real time in the captured video, thereby improving detection efficiency and saving time costs.
[0291] Specifically, the detection type includes one or more of the following:
[0292] The construction safety information corresponding to the object to be detected includes but is not limited to whether the operator wears a safety helmet, insulating gloves, and whether a fire extinguisher is provided;
[0293] The service information corresponding to the object to be detected includes but is not limited to the operator's clothing, work card, work clothes, etc.;
[0294] The device resource information corresponding to the object to be detected includes but is not limited to the installation object of the operator, the model of the device, the serial number (SN) of the device, the label of the device, etc.;
[0295] The construction process information corresponding to the object to be inspected includes but is not limited to whether the waste treatment is compliant, equipment location, grounding wire setting, wiring, etc.
[0296] Of course, in actual applications, the content of the detection type can also be adaptively adjusted according to different application scenarios.
[0297] Step 103: Detect the video and display the detection result of each object to be detected.
[0298] Optionally, the test results of each object to be tested may include but are not limited to: information of the object to be tested (such as name, device model, etc.), the test results of each object to be tested (such as whether it passed the test), information used to indicate the progress of the test (such as whether the test is completed), etc.
[0299] In an embodiment of the present application, the video is detected in real time during the shooting process. That is, the detection is performed while the video is being acquired, and the detection result of each object to be detected is displayed by the first electronic device. In this way, the acquisition of the detection result can be made more timely, so as to facilitate the timely location of the problem or fault. Depending on the object to be detected, there may also be different detection methods, such as optical character recognition (OCR), image analysis, and detection using artificial intelligence (AI) models.
[0300] The detection of the video may include one or more of the following methods:
[0301] Method 1: The video is detected by the first electronic device.
[0302] In practical applications, the first electronic device can perform detection through the deployed AI model. Considering factors such as the resources of the first electronic device, the AI model deployed on the first electronic device side can meet requirements such as lightweight and fast reasoning speed.
[0303] During the detection process, the first electronic device selects a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected. Afterwards, detection can be performed based on the information of the target object to be detected, and the detection result of the target object to be detected can be displayed. In this way, by deploying the model on the first electronic device, the content in the image can be obtained in real time even in the absence of a network, and the determination of the detection type or content can be completed. At the same time, the detection of the first electronic device can also be completed.
[0304] The frames to be detected can be obtained in a variety of ways, for example, they can be obtained at a predetermined time interval or at a predetermined frame number interval. The time interval and frame number interval can be set as required. Optionally, the acquired frames to be detected can also be pre-processed to speed up subsequent processing, for example, deleting images that do not meet the definition requirements, resizing the image frames, etc.
[0305] One or more objects to be detected can be obtained by detecting the frame to be detected. Here, a target object to be detected can be selected from the acquired objects to be detected. Among them, the confidence of the target object to be detected in the frame to be detected is greater than or equal to the first preset value, or the confidence of the target object to be detected in the consecutive N frames to be detected is greater than or equal to the second preset value, N is an integer greater than 1; or the display ratio of the target object to be detected in the frame to be detected is greater than or equal to the third preset value, or the display ratio of the target object to be detected in the consecutive M frames to be detected is greater than or equal to the fourth preset value, M is an integer greater than 1.
[0306] Among them, the values of the first preset value, the second preset value, N and M can be set as needed. The confidence level indicates the possibility of an object being accurately identified, and the display ratio can be understood as the ratio between the display area of the object to be detected in the frame to be detected and the size of the frame to be detected. The higher the confidence level or the larger the display ratio, the greater the possibility that the object to be detected is accurately identified. By judging the confidence level or the display ratio, the problem of interference identification caused by the content that the camera has unintentionally scanned can be solved, and invalid work can be avoided, thereby ensuring the reliability of the detection results and improving the detection efficiency.
[0307] For example, the frames to be detected can be identified to obtain multiple objects to be detected. The confidence of each object to be detected is obtained. If the confidence of one of the objects to be detected is greater than a first preset value, the object to be detected can be used as a target object to be detected. Alternatively, further, if the confidence of the object to be detected in N consecutive frames to be detected is greater than the first preset value, the object to be detected can be used as a target object to be detected. In this embodiment of the present application, the method for obtaining the confidence is not limited.
[0308] For example, the frame to be detected can be identified to obtain multiple objects to be detected. The display ratio of each object to be detected in the frame to be detected is obtained. If the display ratio of a certain object to be detected in the frame to be detected is greater than the second preset value, the object to be detected can be used as the target object to be detected. Alternatively, further, if the display ratio of the object to be detected in the continuous M frames to be detected is greater than the second preset value, the object to be detected can be used as the target object to be detected.
[0309] Through the above judgment on the confidence level or the display ratio, the information of the target object to be detected can also be obtained, including any of the following:
[0310] The frame to be detected;
[0311] The identification (such as ID, etc.) of the frame to be detected and the target object to be detected;
[0312] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0313] Among them, if only the confidence or display ratio of the object to be detected in a certain frame is judged, the frame to be detected may refer to the frame of image in which the confidence of the object to be detected is greater than the first preset value or the display ratio is greater than the second preset value. If the confidence of the object to be detected in N consecutive frames of images or the display ratio in M consecutive frames of images is judged, then the frame to be detected may be any frame in the N consecutive frames to be detected or the M consecutive frames to be detected that meet the above conditions, such as the Nth frame or the Mth frame.
[0314] By acquiring the information of the target object to be detected, it is convenient to display the detection results after obtaining them, so that users can intuitively understand the detection results of each object to be detected. At the same time, if cloud detection is required, the detection efficiency of the cloud can be improved by sending the information of the target object to be detected to the cloud.
[0315] Method 2: Detect the video through the cloud.
[0316] In this method, the first electronic device sends the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected. The information to be detected is determined based on a video including at least two objects to be detected. For example, the information to be detected includes the video, or the information of the aforementioned target object to be detected. That is to say, the first electronic device can send the acquired video to the cloud for processing, or send the information of the target object to be detected obtained in accordance with the aforementioned method one to the cloud for processing. Then, for the latter method, the bandwidth occupancy can be reduced, and the detection efficiency and speed of the cloud can be improved.
[0317] In the second method, the first electronic device may also send detection task information to the cloud, wherein the detection task information is used to determine the target processing model, and the target processing model is used to detect the information to be detected. The detection task information is used to indicate to the cloud what type of detection to perform, that is, to indicate the type of detection of the target object to be detected, so as to facilitate detection in the cloud and improve the detection speed. Among them, the target processing model can be divided into multiple types, including but not limited to OCR model, small model, large model, etc.
[0318] Method three: detecting the video through the first electronic device and detecting the video through the cloud.
[0319] In this way, the first electronic device and the cloud coordinate to detect the video. For example, the first electronic device can determine whether the detection of certain objects to be detected is performed by the cloud or by the first electronic device based on one or more factors such as its own capability information, detection configuration information, and the detection complexity of the objects to be detected. Among them, the detection configuration information may include the correspondence between the objects to be detected and the detection method, and the detection method may include detection on the first device side and detection on the cloud.
[0320] For example, if a certain inspection task is to detect certain key factors to pass the inspection, such as wearing a safety helmet, work card, etc., at this time, you only need to use the capabilities of the terminal side (that is, the first electronic device side) to detect the key factors. This type of inspection item belongs to the inspection that can be completed on the first electronic device side; if some inspection tasks need to further determine the content or judge whether it is qualified through cognition after detecting the key factors, for example, after the terminal side model detects the device label, it is necessary to determine whether the device information is consistent with the preset information. At this time, it is necessary to call the cloud's OCR capability for the label to confirm whether it is qualified. Therefore, this type of inspection item needs to use cloud detection.
[0321] In this way, for detection performed by the first electronic device, the first electronic device uses its own AI model to perform detection. For detection performed by the cloud, the first electronic device can send the obtained target detection object information and detection task information to the cloud, and the cloud will perform detection.
[0322] In the above methods 2 and 3, the processing power of the cloud can be fully utilized, and detection can be achieved through the collaboration of large and small models of different types or capabilities, realizing the automatic optimization of recognition capabilities, thereby improving the efficiency and speed of detection. At the same time, relying on the powerful understanding and reasoning capabilities of large models, for new detection requirements, a small number of samples or zero sample recognition can be achieved, which can quickly support business needs.
[0323] In the embodiment of the present application, the display order of the detection results satisfies at least one of the following:
[0324] 1. The display order of the detection results is consistent with the shooting order of the objects to be detected in the video.
[0325] For example, for multiple objects A, B, and C to be detected, the shooting order is in the order of A, B, and C. Then, when displaying the detection results, the detection results of A, B, and C are displayed in the order of A, B, and C respectively.
[0326] 2. The display order of the detection results is consistent with the order in which the objects to be detected are detected.
[0327] For example, in the video, objects A, B, and C are detected in sequence. When the detection results are displayed, the detection results of A, B, and C are displayed in the order of A, B, and C, respectively.
[0328] 3. When the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of the preset detection ranges.
[0329] The detection range may refer to the range of the object to be detected in the detection environment where the object to be detected is located, or may refer to the detection range where the object to be detected is located in a certain frame image of a video, etc. For example, the detection range where the object to be detected is located may be divided into several parts, and the order between these parts may be set. For example, the frame to be detected may also be divided into several parts, and the order between these parts may be set.
[0330] Taking the range of the object to be detected in the detection environment where the object to be detected is located as an example, a detection sequence of different detection ranges can be set. Therefore, according to the detection sequence, the detection results of the object to be detected in different detection ranges can be displayed.
[0331] For example, in this video, it is detected that the objects A and B to be detected are located in ranges F1 and F2 respectively, and the shooting order of the two is the same. The preset order of detection ranges is F1 and F2. Then, when displaying the detection results, the detection results of A and B are displayed in the order of A and B.
[0332] By displaying the test results of each object to be tested in different display orders, the intuitiveness of the displayed results can be improved, making it easier for users to compare and verify.
[0333] Based on the above description, it can be seen that in the embodiment of the present application, the first electronic device, when obtaining the first input for shooting a video, shoots a video, and detects the video, and displays the detection result of each object to be detected in the video. Since it is based on the video in the embodiment of the present application to detect, and the video includes at least two objects to be detected and the at least two objects to be detected correspond to different detection types, therefore, the scheme of the embodiment of the present application can simultaneously detect different detection types of multiple objects to be detected in the video, thereby avoiding the problem of low detection efficiency caused by the need to repeatedly shoot a picture of a certain detection object and detect the picture in the prior art, and then, the scheme of the embodiment of the present application can improve the detection efficiency. At the same time, since the detection in the embodiment of the present application is based on the video obtained, therefore, it is not necessary for the operator to shoot a large number of pictures, thereby the workload of the operator can also be reduced.
[0334] See also Figure 2 , Figure 21 is a flow chart of a video detection method provided in an embodiment of the present application, which is applied to a second electronic device. The second electronic device is applied to the cloud, such as a server. Figure 2 As shown, the following steps are included:
[0335] Step 201: Acquire information to be detected, wherein the information to be detected is acquired based on a video shot by a first electronic device and includes at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0336] The information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected. The meaning of the information of the target object to be detected can refer to the description of the above embodiment.
[0337] If the information to be detected includes a video, the second electronic device can extract the required information from the video in the same manner as the first electronic device, such as the information of the target object to be detected. If the information to be detected includes the information of the target object to be detected, the second electronic device can directly perform detection based on the obtained information of the target object to be detected.
[0338] Step 202: Perform detection based on the information to be detected to obtain a detection result for each object to be detected.
[0339] Here, the second electronic device can be detected in different ways.
[0340] Method 1: The second electronic device uses a target processing model to detect the information to be detected to obtain a detection result for each object to be detected; wherein the target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0341] That is, the second electronic device can be detected by the model deployed by itself. The first type of model and the second type of model mentioned above can also be referred to as a small model and a large model, respectively, wherein the small model and the large model are defined as follows:
[0342] Small models are mainly used to complete tasks related to text detection and detailed device model recognition in the detection type, such as label text recognition, device model recognition, SN code recognition, etc. Small models are usually mature existing models or capabilities, and the recognition rate can meet business requirements.
[0343] The large model is mainly used to complete complex detection scenarios that require contextual reasoning in the detection type, such as motion detection, installation process detection, etc. Based on the characteristics of the large model itself, when there is a new detection type, it can also be identified based on fine-tuning the large model.
[0344] Then, in this case, the second electronic device can determine the corresponding target processing model according to the information of the target object to be detected. When determining the target processing model, capabilities can be arranged and different capabilities can be selected according to the complexity of the detection. When selecting the corresponding target processing model, the general principle is: for special detection items, such as label OCR recognition, device model recognition and other detection items, use a small model for recognition; and for complex detection scenarios such as action detection and installation process that require contextual reasoning, or multiple detection items contained in one picture, a large model can be used for recognition.
[0345] In the embodiments of the present application, through the combined use of large models and small models, for new quality inspection needs, by selecting a large model, it is possible to rely on the large model for fine-tuning through a small number of samples or even zero samples, so that new detection functions can be quickly realized. At the same time, the terminal APP does not need to perform interface development and adaptation, and flexible configuration of quality inspection tasks of different business types can be achieved.
[0346] In a second method, the second electronic device sends information of a target processing model and the information to be detected to a third electronic device, and the third electronic device detects the information to be detected using the target processing model to obtain a detection result of each object to be detected.
[0347] That is to say, in this way, the second electronic device sends the information of the target processing model and the information to be detected to the third electronic device, and the third electronic device performs the detection of the information to be detected. The second electronic device can determine the target processing model in the manner described in the first mode above and send it to the third electronic device. The information of the target processing model can be, for example, the identification and name of the model. The third electronic device can be an electronic device set in the cloud, such as a server.
[0348] In both of the above methods, the second electronic device can determine the detection type by its own judgment of the information to be detected, detection configuration information, etc. Alternatively, in an embodiment of the present application, the second electronic device can also obtain detection task information from the first electronic device, wherein the detection task information is used to determine the target processing model. Thus, in this way, the processing burden of the second electronic device can be reduced, thereby speeding up the processing speed of the second electronic device.
[0349] For example, the second electronic device may determine the detection type according to the detection task information, and determine the corresponding target processing model according to the correspondence between the detection type and the model. The correspondence between the detection type and the model records the correspondence between different detection types and models. Of course, the correspondence can be expanded or adjusted as needed. Alternatively, after determining the detection type, the second electronic device may select the corresponding target processing model according to the above-mentioned general principle of selecting the corresponding target processing model.
[0350] Step 203: Send the detection result of each object to be detected to the first electronic device.
[0351] The test result satisfies at least one of the following:
[0352] 1. The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot.
[0353] For example, for multiple objects A, B, and C to be detected, it is determined through video analysis that they are shot in the order of A, B, and C. Then, when sending the detection results, the detection results of A, B, and C are sent in the order of A, B, and C respectively.
[0354] 2. The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected.
[0355] For example, in the video, objects A, B, and C are detected in sequence. When sending the detection results, the detection results of A, B, and C are sent in the order of A, B, and C respectively.
[0356] 3. When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges.
[0357] For example, in this video, it is detected that the objects A and B to be detected are located in ranges F1 and F2 respectively, and the shooting order of the two is the same. The preset order of detection ranges is F1 and F2. Then, when sending the detection results, the detection results of A and B are sent in the order of A and B.
[0358] 4. The detection result includes the detection results of some or all of the at least two objects to be detected.
[0359] That is to say, the second electronic device can send the detection results of one or more objects to be detected at the same time.
[0360] By sending the detection results of each object to be detected in different sending orders, the receiving end can improve the intuitiveness of the displayed results, which is convenient for users to compare and verify.
[0361] In the embodiment of the present application, since the detection is performed based on video in the embodiment of the present application, and the video includes at least two objects to be detected and the at least two objects to be detected correspond to different detection types, the scheme of the embodiment of the present application can simultaneously perform different detection types on multiple objects to be detected in the video, thereby avoiding the problem of low detection efficiency caused by the prior art that only pictures of a certain detection object can be taken according to the detection requirements and the pictures can be detected. Furthermore, the detection efficiency can be improved by utilizing the scheme of the embodiment of the present application.
[0362] See also Figure 3 , Figure 3 1 is a flow chart of a video detection method provided in an embodiment of the present application, which is applied to a third electronic device. The third electronic device is applied to the cloud, such as a server. Figure 3 As shown, the following steps are included:
[0363] Step 301, receiving target processing model information and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video shot by a first electronic device and includes at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0364] The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model. The information of the target processing model may be, for example, the identification and name of the model. The first type of model and the second type of model may also be referred to as a small model and a large model, respectively, wherein the small model and the large model are defined as follows:
[0365] Small models are mainly used to complete tasks related to text detection and detailed device model recognition in the detection type, such as label text recognition, device model recognition, SN code recognition, etc. Small models are usually mature existing models or capabilities, and the recognition rate can meet business requirements.
[0366] The large model is mainly used to complete complex detection scenarios that require contextual reasoning in the detection type, such as motion detection, installation process detection, etc. Based on the characteristics of the large model itself, when there is a new detection type, it can also be identified based on fine-tuning the large model.
[0367] Step 302: Use the target processing model to detect the information to be detected to obtain a detection result for each object to be detected.
[0368] The meaning of the information to be detected can refer to the description of the above method embodiment. The third electronic device can call the target processing model according to the information of the target processing model, and perform detection to obtain the detection result of each object to be detected.
[0369] Step 303: Send the detection result of each object to be detected to the first electronic device.
[0370] The test result satisfies at least one of the following:
[0371] 1. The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot.
[0372] For example, for multiple objects A, B, and C to be detected, the video analysis determines that the shooting order is in the order of A, B, and C. Then, when sending the detection results, the detection results of A, B, and C are sent in the order of A, B, and C respectively.
[0373] 2. The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected.
[0374] For example, in the video, objects A, B, and C are detected in sequence. When sending the detection results, the detection results of A, B, and C are sent in the order of A, B, and C respectively.
[0375] 3. When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges.
[0376] For example, in this video, it is detected that the objects A and B to be detected are located in ranges F1 and F2 respectively, and the shooting order of the two is the same. The preset order of detection ranges is F1 and F2. Then, when sending the detection results, the detection results of A and B are sent in the order of A and B.
[0377] 4. The detection result includes the detection results of some or all of the at least two objects to be detected.
[0378] That is to say, the third electronic device can send the detection results of one or more objects to be detected at the same time.
[0379] By sending the detection results of each object to be detected in different sending orders, the receiving end can improve the intuitiveness of the displayed results, which is convenient for users to compare and verify.
[0380] In the embodiment of the present application, since the detection is performed based on video in the embodiment of the present application, and the video includes at least two objects to be detected and the at least two objects to be detected correspond to different detection types, the scheme of the embodiment of the present application can simultaneously perform different detection types on multiple objects to be detected in the video, thereby avoiding the problem of low detection efficiency caused by the prior art that only pictures of a certain detection object can be taken according to the detection requirements and the pictures can be detected. Furthermore, the detection efficiency can be improved by utilizing the scheme of the embodiment of the present application.
[0381] In an embodiment of the present application, the second electronic device and the third electronic device may also be functions or modules located in the same device or equipment. Then, at this time, the interaction between the second electronic device and the third electronic device becomes the interaction between different functions or modules in the device or equipment. In this case, the second electronic device is responsible for obtaining the corresponding information to be detected from the first electronic device, and determining the target processing model, sending the information of the target processing model and the information to be detected to the third electronic device, and the third electronic device uses the target processing model to detect the information to be detected, obtains the detection result of each object to be detected, and sends the corresponding detection result to the first electronic device.
[0382] The specific implementation process of the embodiment of the present application is described in detail below. The first electronic device is a terminal as an example, and the second electronic device and the third electronic device are both set on a server in the cloud, as two modules of the server interacting with each other. Figure 4 , the processing of the terminal may include the following steps:
[0383] Step 401: The terminal receives the user's input, starts the camera to shoot, and obtains the video.
[0384] like Figure 5 As shown, multiple tests can be provided on the display interface of the terminal for users to choose, such as test A, test B, video intelligent quality inspection, etc. For example, when the user selects "video intelligent quality inspection", the camera can be started and start shooting each object in the environment to be tested. Among them, when shooting the video, the shooting order of each object is random. Figure 6 As shown, the objects to be detected currently included in the captured video are a helmet and a work card of a certain operator.
[0385] Step 402: The terminal extracts frames from the real-time images in the acquired video to obtain frames to be detected.
[0386] The extracted images may also be preprocessed to meet the input requirements of the terminal side model. The preprocessing may include deleting images whose clarity does not meet the requirements, adjusting the size of the images, etc. The specific method of extracting frames is not limited here. For example, video frames may be extracted from the video at a preset time interval as frames to be detected.
[0387] Step 403: The terminal identifies the input frame to be detected and obtains the object to be detected.
[0388] For the recognition of an image, one or more objects to be detected may be obtained. If there are multiple objects to be detected, the recognition result can be expressed in the form of an array. Among them, the recognition result of each object to be detected may include the object to be detected, the position information of the object to be detected and the confidence level. Among them, the position information of the object to be detected refers to the position of the object to be detected in the frame to be detected, which can be shown in the form of a box or the like.
[0389] Step 404: The terminal judges the validity of the recognition result of each frame. If the confidence of a certain object to be detected in N consecutive (for example, N=3) frames is greater than the first preset threshold value C, then the object to be detected can be used as the target object to be detected. Otherwise, the object to be detected can be used as an invalid recognition result.
[0390] Among them, the effective recognition results can be directly rendered in the picture, including the location of the detected object, and the recognized content and the corresponding image can also be displayed. Figure 7 As shown, the target object to be detected is a label. In the display interface of the terminal, 71 indicates the position of the label in the image. At the same time, the display interface shows that the recognition result is a label. Figure 6 As shown, in Figure 8 In the display interface shown, the target objects to be detected are a safety helmet and a work card, and 81 and 82 indicate the positions of the safety helmet and the work card in the image.
[0391] Step 405: The terminal analyzes the target object to be detected and determines whether to perform detection on the terminal side or on the cloud side.
[0392] The terminal can determine the detection type corresponding to the target object to be detected according to the pre-built detection type association table (as shown in Table 1), and judge the detection type to determine whether the detection is performed by the terminal side or by the cloud.
[0393] For example, the terminal may determine whether the detection of certain objects to be detected is performed by the cloud or by the terminal based on one or more factors such as its own capability information, detection configuration information, and the detection complexity of the objects to be detected.
[0394] For example, if a certain detection type is qualified when certain key factors are detected, such as wearing a safety helmet, work card, etc., at this time, you only need to use the capabilities of the end side (i.e., the terminal side) to detect the key factors. This type of detection belongs to the detection that can be completed on the terminal side; if some detection types need to further determine their content or judge whether they are qualified through cognition after detecting the key factors, for example, after the end-side model detects the device label, it is necessary to determine whether the device information is consistent with the preset information. At this time, it is necessary to call the cloud's OCR capability for the label to confirm whether it is qualified. Therefore, this type of detection requires the use of cloud detection.
[0395] Since each detection type contains a key factor, a detection type association table (as shown in Table 1) can be pre-built to indicate the relationship between the detection type and the key factor. At the same time, the detection type can be simply classified into two categories: detection that can be completed on the terminal side and detection that cannot be completed on the terminal side. In addition, based on the recognition results of the terminal side model, combined with Table 1, the specific task list that needs to be detected can be determined.
[0396] In Table 1, the objects to be detected can be mapped to various categories based on the objects and the division dimensions. At the same time, the objects to be detected can be mapped to specific detection types according to Table 1. In combination with Table 1, it is also identified which detections can be completed by the terminal and which detections need to be performed in the cloud (i.e., the detections corresponding to the objects to be detected marked as "No" in the "Detection can be completed on the terminal side" column in Table 1).
[0397] Table 1
[0398]
[0399] It should be noted that the above Table 1 is only an example of information such as the object to be detected and the detection type of the object to be detected. In practical applications, the content of Table 1 can be adjusted as needed.
[0400] Step 406: If the detection is performed by the terminal side, the AI model on the terminal side may be called for detection. If the detection is performed by the cloud, the corresponding information to be detected may be sent to the cloud, such as the frame to be detected, the identifier of the target object to be detected, the location information of the target object to be detected in the frame to be detected, and the detection task information. The information to be detected may be sent to the cloud in the form of a list.
[0401] Among them, the AI model on the terminal side is mainly used to complete the detection of targets related to the detection items in the captured video images, such as safety helmets, work clothes, work cards, equipment, labels and other targets, which belongs to a multi-target detection model.
[0402] For example, Figure 7In the process, it involves the detection of labels, including whether the labels are posted, whether the labels are consistent with the preset labels, etc. Whether the labels are posted can be considered as the detection of construction process information, which can be performed by the terminal side; and whether the labels are consistent with the preset labels can be considered as the detection of equipment resource information, which requires the use of OCR capabilities and can therefore be detected by the cloud. For example, in Figure 8 In the process, the detection of safety cards and work cards belongs to the detection of construction safety information and service information respectively, which can be executed by the terminal.
[0403] Recombination Figure 4 , the cloud handles this as follows:
[0404] Step 407: After receiving the information to be detected sent by the terminal, the capability arrangement module in the cloud (corresponding to the aforementioned second electronic device) selects the corresponding target processing model or capability according to the pre-configured capability arrangement design table (as shown in Table 2).
[0405] The capability arrangement module arranges capabilities according to the capability arrangement design table and selects different models or capabilities according to the complexity of the detection items (i.e., detection types). When selecting models, the general principle is to use small models for special detection items, such as label OCR recognition, device model recognition, etc.; while complex detection scenarios such as action detection and installation process that require contextual reasoning, or multiple detection items in one image, can be identified using large models.
[0406] Table 2 shows the correspondence between different detection types and models or capabilities. In practical applications, the content of Table 2 can be adjusted according to actual needs.
[0407] Table 2
[0408]
[0409] Step 408: The capability orchestration module in the cloud sends the information of the determined target processing model and the information to be detected to the intelligent quality inspection module in the cloud (corresponding to the aforementioned third electronic device).
[0410] Step 409: The intelligent quality inspection module in the cloud calls the corresponding processing model to inspect the information to be inspected.
[0411] Step 410: The intelligent quality inspection module in the cloud returns the inspection result to the terminal.
[0412] Among them, the intelligent quality inspection module in the cloud can encapsulate the test results into a structured data sequence and return it to the terminal, thereby improving the speed and efficiency of sending the results.
[0413] On the terminal side, the content of the detection results displayed may be different depending on the different objects to be detected.
[0414] For example, the schematic diagram of the test results displayed on the terminal interface is as follows Fig. 9 As shown. Fig. 9 The test progress (such as completion of the test), the overall test results (the figure shows that all test items have been tested successfully), and the information on test passes, such as the test results of the safety helmet and work card (all passed the test in this figure).
[0415] For example, the schematic diagram of the test results displayed on the terminal interface is as follows Fig.10 As shown. Fig.10 In the figure, the progress of the test (such as the completion of the test, the number of items that have been tested (3 items are shown in the figure)), the test results of the SN code, device model, and label (all passed the test in this figure) are displayed. Optionally, the image including the SN code, device model, and label indicated by 1001 may also be included.
[0416] For example, the schematic diagram of the test results displayed on the terminal interface is as follows Fig.11 As shown. Fig.11 In the figure, the progress of the inspection (such as the completion of the inspection, the number of items completed (3 items are shown in the figure)), the inspection results of the helmet, equipment model, and label (all passed the inspection in this figure) are displayed. Optionally, the image of the helmet, equipment model, and label indicated by 1101 may also be included.
[0417] For example, after all detection tasks are completed, the user can click the End button to complete the shooting and recognition and jump to the acceptance result page (such as Fig.12 This page contains all the detection objects and detection results during the one-shot shooting process, and may also include the corresponding frames to be detected. Figure 6 The detection of the safety helmet and the work card shown in the figure may include whether the detection of the safety helmet is completed, whether the detection of the work card is completed, whether the safety helmet is worn, whether the work card is worn, whether the information of the work card is consistent with the preset operator information, etc. At the same time, the user can click the submit button, and the detection result will be sent to the corresponding system receiving the detection result.
[0418] It can be seen from the above description that for the video formed by the original image obtained by the terminal each time between the start and end of shooting, compared with the existing solution that requires solidification of the detection range and sequence, the detection is flexible and efficient, and can better adapt to the needs of different businesses. At the same time, using the solution of the embodiment of the present application, the intelligent recognition analysis of the detection object and the automatic arrangement of the model or capability can be realized. Since the terminal-side model can realize part of the detection task, the capacity load of the cloud is reduced, and the terminal transmits the information of the target object to be detected as effective content to the cloud for detection, thereby reducing the bandwidth occupancy and improving the detection efficiency.
[0419] The embodiments of the present application can be applied to scenarios where quality inspection of installation and maintenance services, safety, equipment resource information, and processes are required for governments, enterprises, institutions, etc. It can also be applied to scenarios where equipment management entities such as governments, enterprises, institutions, etc. need to audit and manage the equipment information they maintain.
[0420] See also Fig.13 , Fig.13 is a structural diagram of a video detection device provided in an embodiment of the present application, which is applied to a first electronic device. Fig.13 As shown, the video detection device includes:
[0421] The first acquisition module 1301 is used to acquire a first input; the first processing module 1302 is used to shoot the video in response to the first input; the second processing module 1303 is used to detect the video and display the detection result of each object to be detected;
[0422] The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0423] Optionally, the video is an original image captured by the first electronic device during a period from the start to the end of each shooting.
[0424] Optionally, the detection type includes one or more of the following:
[0425] Construction safety information corresponding to the object to be detected;
[0426] Service information corresponding to the object to be detected;
[0427] The device resource information corresponding to the object to be detected;
[0428] The construction process information corresponding to the object to be detected.
[0429] Optionally, during the process of shooting the video, the shooting order of the objects to be detected is random.
[0430] Optionally, the first processing module 1302 is further configured to perform real-time detection on the video during shooting.
[0431] Optionally, the display order of the detection results satisfies at least one of the following:
[0432] The display order of the detection results is consistent with the shooting order of the objects to be detected in the video;
[0433] The display order of the detection results is consistent with the order in which the objects to be detected are detected;
[0434] In the case where the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of preset detection ranges.
[0435] Optionally, the second processing module 1303 is further configured to:
[0436] Detecting the video by the first electronic device;
[0437] The video is detected through the cloud.
[0438] Optionally, the second processing module 1303 is further used to:
[0439] Selecting a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected;
[0440] Performing detection based on the information of the target object to be detected, and displaying the detection result of the target object to be detected;
[0441] The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value, or the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0442] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0443] Optionally, the information of the target object to be detected includes any of the following:
[0444] The frame to be detected;
[0445] Identifications of the frame to be detected and the target object to be detected;
[0446] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0447] Optionally, the second processing module 1303 is further used to:
[0448] Sending the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected;
[0449] The information to be detected is determined based on a video including at least two objects to be detected.
[0450] Optionally, the information to be detected includes the video, or information of the target object to be detected.
[0451] Optionally, the device may further include:
[0452] A first sending module, used for sending detection task information to the cloud;
[0453] The detection task information is used to determine a target processing model, and the target processing model is used to detect the information to be detected.
[0454] Optionally, the first electronic device is a user terminal device.
[0455] The device provided in the embodiment of the present application can execute the above-mentioned method embodiment applied to the first electronic device. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.
[0456] See also Fig.14 , Fig.14 is a structural diagram of a video detection device provided in an embodiment of the present application, which is applied to a second electronic device. Fig.14 As shown, the video detection device includes:
[0457] The first acquisition module 1401 is used to acquire information to be detected, wherein the information to be detected is acquired based on a video including at least two objects to be detected shot by a first electronic device, and the at least two objects to be detected correspond to different detection types; the second acquisition module 1402 is used to perform detection based on the information to be detected to obtain a detection result for each object to be detected; the first sending module 1403 is used to send the detection result of each object to be detected to the first electronic device.
[0458] Optionally, the second obtaining module 1402 is further configured to:
[0459] Using the target processing model to detect the information to be detected, and obtaining the detection result of each object to be detected;
[0460] The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0461] Optionally, the second obtaining module 1402 is further configured to:
[0462] Sending information of the target processing model and the information to be detected to a third electronic device, wherein the third electronic device is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0463] The target processing model includes a first type of model or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0464] Optionally, the device may further include:
[0465] The third acquisition module is used to acquire detection task information, wherein the detection task information is used to determine the target processing model.
[0466] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0467] Optionally, the information of the target object to be detected includes any of the following:
[0468] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0469] Identifications of the frame to be detected and the target object to be detected;
[0470] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0471] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0472] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0473] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0474] Optionally, the test result satisfies at least one of the following:
[0475] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0476] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0477] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0478] The detection result includes detection results of some or all of the at least two objects to be detected.
[0479] Optionally, the second electronic device is located in the cloud.
[0480] The device provided in the embodiment of the present application can execute the above-mentioned method embodiment applied to the second electronic device. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.
[0481] See also Fig.15 , Fig.15 is a structural diagram of a video detection device provided in an embodiment of the present application, which is applied to a third electronic device. Fig.15 As shown, the video detection device includes:
[0482] The first receiving module 1501 is used to receive information of a target processing model and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected shot by the first electronic device, and the at least two objects to be detected correspond to different detection types; the first acquiring module 1502 is used to detect the information to be detected using the target processing model to obtain a detection result for each object to be detected; the first sending module 1503 is used to send the detection result of each object to be detected to the first electronic device.
[0483] Optionally, the target processing model includes: a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than the processing capability of the first type of model.
[0484] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0485] Optionally, the information of the target object to be detected includes any of the following:
[0486] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0487] Identifications of the frame to be detected and the target object to be detected;
[0488] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0489] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0490] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1.
[0491] Optionally, the test result satisfies at least one of the following:
[0492] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0493] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0494] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0495] The detection result includes detection results of some or all of the at least two objects to be detected.
[0496] Optionally, the third electronic device is located in the cloud.
[0497] The device provided in the embodiment of the present application can execute the above-mentioned method embodiment applied to the third electronic device. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.
[0498] See also Fig.16 , Fig.16 is a structural diagram of a video detection device provided in an embodiment of the present application, which is applied to a first electronic device. Fig.16 As shown, the video detection device includes: a processor 1601 and a transceiver 1602;
[0499] The transceiver 1602 is used to obtain a first input;
[0500] The processor 1601 is configured to shoot the video in response to the first input; detect the video and display the detection result of each object to be detected;
[0501] The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
[0502] Optionally, the video is an original image captured by the first electronic device during a period from the start to the end of each shooting.
[0503] Optionally, the detection type includes one or more of the following:
[0504] Construction safety information corresponding to the object to be detected;
[0505] Service information corresponding to the object to be detected;
[0506] The device resource information corresponding to the object to be detected;
[0507] The construction process information corresponding to the object to be detected.
[0508] Optionally, during the process of shooting the video, the shooting order of the objects to be detected is random.
[0509] Optionally, the processor 1601 is also used to perform real-time detection on the video during shooting.
[0510] Optionally, the display order of the detection results satisfies at least one of the following:
[0511] The display order of the detection results is consistent with the shooting order of the objects to be detected in the video;
[0512] The display order of the detection results is consistent with the order in which the objects to be detected are detected;
[0513] In the case where the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of preset detection ranges.
[0514] Optionally, the processor 1601 is further configured to perform at least one of the following:
[0515] Detecting the video by the first electronic device;
[0516] The video is detected through the cloud.
[0517] Optionally, the processor 1601 is further configured to:
[0518] Selecting a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected;
[0519] Performing detection based on the information of the target object to be detected, and displaying the detection result of the target object to be detected;
[0520] The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value, or the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0521] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0522] Optionally, the information of the target object to be detected includes any of the following:
[0523] The frame to be detected;
[0524] Identifications of the frame to be detected and the target object to be detected;
[0525] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0526] Optionally, the processor 1601 is further configured to:
[0527] Sending the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected;
[0528] The information to be detected is determined based on a video including at least two objects to be detected.
[0529] Optionally, the information to be detected includes the video, or information of the target object to be detected.
[0530] Optionally, the processor 1601 is further configured to:
[0531] Sending detection task information to the cloud;
[0532] The detection task information is used to determine a target processing model, and the target processing model is used to detect the information to be detected.
[0533] Optionally, the first electronic device is a user terminal device.
[0534] The device provided in the embodiment of the present application can execute the above-mentioned method embodiment applied to the first electronic device. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.
[0535] See also Fig.17 , Fig.17 is a structural diagram of a video detection device provided in an embodiment of the present application, which is applied to a second electronic device. Fig.17 As shown, the video detection device includes: a processor 1701 and a transceiver 1702;
[0536] The transceiver 1702 is used to obtain information to be detected, wherein the information to be detected is obtained based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0537] The processor 1701 is used to perform detection based on the information to be detected, and obtain a detection result of each object to be detected;
[0538] The transceiver 1702 is further configured to send the detection result of each object to be detected to the first electronic device.
[0539] Optionally, the processor 1701 is further configured to:
[0540] Using the target processing model to detect the information to be detected, and obtaining the detection result of each object to be detected;
[0541] The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0542] Optionally, the processor 1701 is further configured to:
[0543] Sending information of the target processing model and the information to be detected to a third electronic device, wherein the third electronic device is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected;
[0544] The target processing model includes a first type of model or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
[0545] Optionally, the processor 1701 is further configured to:
[0546] Acquire detection task information, wherein the detection task information is used to determine the target processing model.
[0547] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0548] Optionally, the information of the target object to be detected includes any of the following:
[0549] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0550] Identifications of the frame to be detected and the target object to be detected;
[0551] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0552] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0553] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or
[0554] The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
[0555] Optionally, the test result satisfies at least one of the following:
[0556] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0557] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0558] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0559] The detection result includes detection results of some or all of the at least two objects to be detected.
[0560] Optionally, the second electronic device is located in the cloud.
[0561] The device provided in the embodiment of the present application can execute the above-mentioned method embodiment applied to the second electronic device. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.
[0562] See also Fig.18 , Fig.18 is a structural diagram of a video detection device provided in an embodiment of the present application, which is applied to a third electronic device. Fig.18 As shown, the video detection device includes: a processor 1801 and a transceiver 1802;
[0563] The transceiver 1802 is used to receive information of a target processing model and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected taken by the first electronic device, and the at least two objects to be detected correspond to different detection types;
[0564] The processor 1801 is used to detect the information to be detected by using the target processing model to obtain a detection result of each object to be detected;
[0565] The transceiver 1802 is further configured to send the detection result of each object to be detected to the first electronic device.
[0566] Optionally, the target processing model includes: a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than the processing capability of the first type of model.
[0567] Optionally, the information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
[0568] Optionally, the information of the target object to be detected includes any of the following:
[0569] The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected;
[0570] Identifications of the frame to be detected and the target object to be detected;
[0571] The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
[0572] Optionally, the confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value;
[0573] Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1.
[0574] Optionally, the test result satisfies at least one of the following:
[0575] The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot;
[0576] The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected;
[0577] When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges;
[0578] The detection result includes detection results of some or all of the at least two objects to be detected.
[0579] Optionally, the third electronic device is located in the cloud.
[0580] The device provided in the embodiment of the present application can execute the above-mentioned method embodiment applied to the third electronic device. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.
[0581] See also Fig.19 , Fig.19 1 is a structural diagram of a video detection system provided in an embodiment of the present application, including: a first electronic device 1901 and a second electronic device 1902, wherein:
[0582] The first electronic device 1901 is used to obtain a first input; in response to the first input, shoot the video; detect the video and display the detection result of each object to be detected; wherein the video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0583] The second electronic device 1902 is used to obtain information to be detected, wherein the information to be detected is obtained based on the video shot by the first electronic device; perform detection based on the information to be detected to obtain a detection result of each object to be detected; and send the detection result of each object to be detected to the first electronic device.
[0584] See also Fig. 20 , Fig. 20 : is a structural diagram of a video detection system provided in an embodiment of the present application, comprising: a first electronic device 2001, a second electronic device 2002 and a third electronic device 2003, wherein:
[0585] The first electronic device 2001 is used to obtain a first input; in response to the first input, shoot the video; detect the video and display the detection result of each object to be detected; wherein the video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types;
[0586] The second electronic device 2002 is used to obtain information to be detected, wherein the information to be detected is obtained based on the video shot by the first electronic device; send information of a target processing model and the information to be detected to the third electronic device, wherein the target processing model is used to detect the information to be detected;
[0587] The third electronic device 2003 is used to receive the information of the target processing model and the information to be detected sent by the second electronic device; use the target processing model to detect the information to be detected to obtain the detection result of each object to be detected; and send the detection result of each object to be detected to the first electronic device.
[0588] The working principle of each device in the system of the embodiment of the present application can refer to the description of the signing method embodiment.
[0589] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0590] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0591] An embodiment of the present application provides a communication device, including: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the video detection method described above.
[0592] The embodiment of the present application also provides a readable storage medium, on which a program is stored, and when the program is executed by the processor, each process of the above-mentioned video detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.
[0593] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned video detection method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they are not repeated here.
[0594] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0595] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.
[0596] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A video detection method, characterized in that: Applicable to a first electronic device, comprising: Get the first input; In response to the first input, capturing the video; Detecting the video and displaying the detection result of each object to be detected; The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
2. The method according to claim 1, characterized in that The video is an original image captured by the first electronic device during a period from the start to the end of each shooting.
3. The method according to claim 1, characterized in that The detection types include one or more of the following: Construction safety information corresponding to the object to be detected; Service information corresponding to the object to be detected; The device resource information corresponding to the object to be detected; The construction process information corresponding to the object to be detected.
4. The method according to claim 1, characterized in that: During the process of shooting the video, the shooting order of the objects to be detected is random.
5. The method according to claim 1, characterized in that The detecting of the video includes: The video is detected in real time during the shooting process.
6. The method according to claim 1, characterized in that The display order of the test results satisfies at least one of the following: The display order of the detection results is consistent with the shooting order of the objects to be detected in the video; The display order of the detection results is consistent with the order in which the objects to be detected are detected; In the case where the order of photographing or detecting at least two objects to be detected is consistent, the display order of the detection results is consistent with the sequence of preset detection ranges.
7. The method according to claim 1, characterized in that The detecting of the video includes at least one of the following: Detecting the video by the first electronic device; The video is detected through the cloud.
8. The method according to claim 1 or 7, characterized in that: The detecting the video and displaying the detection result of each object to be detected includes: Selecting a frame to be detected from the video, wherein the frame to be detected includes information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected; Performing detection based on the information of the target object to be detected, and displaying the detection result of the target object to be detected; The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value, or the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
9. The method according to claim 8, characterized in that The information of the target object to be detected includes any of the following: The frame to be detected; Identifications of the frame to be detected and the target object to be detected; The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
10. The method according to claim 8, characterized in that The video is detected through the cloud, including: Sending the information to be detected to the cloud, wherein the cloud is used to detect the information to be detected; The information to be detected is determined based on a video including at least two objects to be detected.
11. The method according to claim 10, characterized in that The information to be detected includes the video, or information of the target object to be detected.
12. The method according to claim 10, characterized in that The method further comprises: Sending detection task information to the cloud; The detection task information is used to determine a target processing model, and the target processing model is used to detect the information to be detected.
13. The method according to claim 1, characterized in that The first electronic device is a user terminal device.
14. A video detection method, characterized in that: Applicable to a second electronic device, comprising: Acquiring information to be detected, wherein the information to be detected is acquired based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types; Perform detection based on the information to be detected to obtain a detection result of each object to be detected; The detection result of each object to be detected is sent to the first electronic device.
15. The method according to claim 14, characterized in that The performing detection based on the information to be detected to obtain the detection result of each object to be detected includes: Using the target processing model to detect the information to be detected, and obtaining the detection result of each object to be detected; The target processing model includes a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
16. The method according to claim 14, characterized in that The performing detection based on the information to be detected to obtain the detection result of each object to be detected includes: Sending information of the target processing model and the information to be detected to a third electronic device, wherein the third electronic device is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected; The target processing model includes a first type of model or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
17. The method according to claim 15 or 16, characterized in that The method further comprises: Acquire detection task information, wherein the detection task information is used to determine the target processing model.
18. The method according to claim 14, characterized in that The information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
19. The method according to claim 18, characterized in that The information of the target object to be detected includes any of the following: The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected; Identifications of the frame to be detected and the target object to be detected; The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
20. The method according to claim 19, characterized in that The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value; Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1; or The display proportion of the target object to be detected in the frame to be detected is greater than or equal to a third preset value, or the display proportion of the target object to be detected in M consecutive frames to be detected is greater than or equal to a fourth preset value, where M is an integer greater than 1.
21. The method according to claim 14, characterized in that The test result satisfies at least one of the following: The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot; The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected; When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges; The detection result includes detection results of some or all of the at least two objects to be detected.
22. The method according to claim 14, characterized in that The second electronic device is located in the cloud.
23. A video detection method, characterized in that: Applicable to third electronic devices, including: Receiving information of a target processing model and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected shot by the first electronic device, and the at least two objects to be detected correspond to different detection types; Using the target processing model to detect the information to be detected, and obtaining a detection result of each object to be detected; The detection result of each object to be detected is sent to the first electronic device.
24. The method according to claim 23, characterized in that The target processing model includes: a first type of model and / or a second type of model, and the processing capability of the second type of model is higher than that of the first type of model.
25. The method according to claim 23, characterized in that The information to be detected includes the video, or information of a target object to be detected, and the target object to be detected is part or all of the at least two objects to be detected.
26. The method according to claim 24, characterized in that The information of the target object to be detected includes any of the following: The frame to be detected of the video, wherein the frame to be detected includes the target object to be detected; Identifications of the frame to be detected and the target object to be detected; The frame to be detected, the identifier of the target object to be detected, and the position information of the target object to be detected in the frame to be detected.
27. The method according to claim 26, characterized in that The confidence level of the target object to be detected in the frame to be detected is greater than or equal to a first preset value; Alternatively, the confidence level of the target object to be detected in N consecutive frames to be detected is greater than or equal to a second preset value, where N is an integer greater than 1.
28. The method according to claim 23, characterized in that The test result satisfies at least one of the following: The order in which the detection results are sent is consistent with the order in which the objects to be detected in the video are shot; The order in which the detection results are sent is consistent with the order in which the objects to be detected are detected; When the order of photographing or detecting at least two objects to be detected is consistent, the order of sending the detection results is consistent with the order of the preset detection ranges; The detection result includes detection results of some or all of the at least two objects to be detected.
29. The method according to claim 23, characterized in that The third electronic device is located in the cloud.
30. A video detection device, characterized in that: Applicable to a first electronic device, comprising: A first acquisition module, used for acquiring a first input; A first processing module, configured to capture the video in response to the first input; A second processing module is used to detect the video and display the detection result of each object to be detected; The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
31. A video detection device, characterized in that: Applicable to a second electronic device, comprising: A first acquisition module, configured to acquire information to be detected, wherein the information to be detected is acquired based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types; A second acquisition module is used to perform detection based on the information to be detected to obtain a detection result of each object to be detected; The first sending module is used to send the detection result of each object to be detected to the first electronic device.
32. A video detection device, characterized in that: Applicable to third electronic devices, including: A first receiving module is used to receive information of a target processing model and information to be detected sent by a second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected taken by the first electronic device, and the at least two objects to be detected correspond to different detection types; A first acquisition module is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected; The first sending module is used to send the detection result of each object to be detected to the first electronic device.
33. A video detection device, characterized in that: Applied to a first electronic device, comprising: a processor and a transceiver; The transceiver is used to obtain a first input; The processor is configured to shoot the video in response to the first input; detect the video and display the detection result of each object to be detected; The video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types.
34. A video detection device, characterized in that: Applied to a second electronic device, comprising: a processor and a transceiver; The transceiver is used to obtain information to be detected, wherein the information to be detected is obtained based on a video taken by the first electronic device and including at least two objects to be detected, and the at least two objects to be detected correspond to different detection types; The processor is used to perform detection based on the information to be detected to obtain a detection result of each object to be detected; The transceiver is further used to send the detection result of each object to be detected to the first electronic device.
35. A video detection device, characterized in that: Applicable to a third electronic device, comprising: a processor and a transceiver; The transceiver is used to receive information of the target processing model and information to be detected sent by the second electronic device, wherein the information to be detected is obtained based on a video including at least two objects to be detected taken by the first electronic device, and the at least two objects to be detected correspond to different detection types; The processor is used to detect the information to be detected using the target processing model to obtain a detection result of each object to be detected; The transceiver is further used to send the detection result of each object to be detected to the first electronic device.
36. A video detection system, characterized in that: include: A first electronic device and a second electronic device, wherein: The first electronic device is used to obtain a first input; in response to the first input, shoot the video; detect the video and display the detection result of each object to be detected; wherein the video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types; The second electronic device is used to obtain information to be detected, wherein the information to be detected is obtained based on the video shot by the first electronic device; perform detection based on the information to be detected to obtain a detection result of each object to be detected; and send the detection result of each object to be detected to the first electronic device.
37. A video detection system, characterized in that: include: A first electronic device, a second electronic device and a third electronic device, wherein: The first electronic device is used to obtain a first input; in response to the first input, shoot the video; detect the video and display the detection result of each object to be detected; wherein the video contains at least two objects to be detected, and the at least two objects to be detected correspond to different detection types; The second electronic device is used to obtain information to be detected, wherein the information to be detected is obtained based on the video shot by the first electronic device; and send information of a target processing model and the information to be detected to the third electronic device, wherein the target processing model is used to detect the information to be detected; The third electronic device is used to receive the information of the target processing model and the information to be detected sent by the second electronic device; use the target processing model to detect the information to be detected to obtain the detection result of each object to be detected; and send the detection result of each object to be detected to the first electronic device.
38. A communication device comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps of the video detection method as described in any one of claims 1 to 29.
39. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the video detection method according to any one of claims 1 to 29 are implemented.
40. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps in the video detection method according to any one of claims 1 to 29.
Citation Information
Patent Citations
Abnormal behavior detection method and device, computer equipment and storage medium
CN113627406A
Self-supervised pre-training target detection method, system and device and storage medium
CN116012658A
Online collaborative industrial visual inspection method and terminal equipment
CN117095273A
Video early warning system, method and equipment based on artificial intelligence detection and medium
CN117612060A
Distributed and contextualized artificial intelligence inference service
US20190042955A1
