Object recognition method, device, equipment, system and storage medium

Abstract

The invention provides an object recognition method, device, equipment and system and a storage medium, and the method comprises the steps: carrying out the photographing of a target position, and obtaining a target video; based on a trained first object identification model, identifying a target object in the target video, and determining label change information of a video frame in the target video according to the target object; adjusting the frame sampling frequency of the environment information acquisition equipment according to the label change information; performing frame sampling processing on the target video according to the frame sampling frequency to obtain a sampling frame, and sending the sampling frame to a cloud server; and receiving model updating information which is fed back by the cloud server and is determined according to the sampling frame, and updating the first object recognition model. According to the invention, there is no need to deploy a complex model in the environment information collection device, the first object recognition model can be updated in time, and the object recognition accuracy of the first object recognition model when data drift occurs is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an object recognition method, device, equipment, system and storage medium. Background Art

[0002] The rapid progress of deep neural networks has promoted the development of real-time video reasoning. Through environmental information collection equipment, it is possible to collect information about vehicles passing through a certain road. For example, in the case of a car accident, when a car accident occurs and video evidence is needed to determine the responsible party, the existing method is to manually retrieve surveillance video recordings for a period of time before and after the accident, which is inefficient and leads to low efficiency in case claims. For video evidence, it can be achieved through video reasoning, but due to the limited computing resources of existing environmental information collection equipment, only lightweight deep neural networks (DNNs) with fewer weights and shallower architectures can be deployed. In video reasoning, due to changes in scenes, objects of the same category will appear visually different. For example, in the process from day to night, the vehicle involved in the car accident is easily inferred as the wrong object. This is the main reason for data drift in the video analysis process, and lightweight DNNs are easily affected by data drift. That is, when the real-time video content changes dramatically, the DNN trained offline has poor accuracy in identifying objects in different environments, which can easily lead to inaccurate recognition results of objects in the video and misidentification of vehicles involved in the car accident. Summary of the invention

[0003] The present application provides an object recognition method, apparatus, device, system and storage medium, aiming to improve the accuracy of object recognition of environmental information acquisition equipment when data drift occurs.

[0004] In a first aspect, the present application provides an object recognition method for an environmental information collection device, the object recognition method comprising the following steps:

[0005] Shoot the target location to obtain the target video;

[0006] Based on the trained first object recognition model, identify the target object in the target video, and determine label change information of the video frame in the target video according to the target object;

[0007] Adjusting the frame sampling frequency of the environmental information acquisition device according to the tag change information;

[0008] Performing frame sampling processing on the target video according to the adjusted frame sampling frequency to obtain a sampling frame, and sending the sampling frame to the cloud server;

[0009] The model update information determined by the cloud server according to the sampling frame is obtained, and the first object recognition model is updated.

[0010] In a second aspect, the present application further provides an object recognition device, the object recognition device comprising:

[0011] A video shooting module is used to shoot a video of the target location to obtain a target video;

[0012] An object recognition module, configured to recognize a target object in the target video based on a trained first object recognition model, and determine label change information of a video frame in the target video according to the target object;

[0013] A frequency adjustment module, used to adjust the frame sampling frequency of the environmental information collection device according to the tag change information;

[0014] A sampling processing module, used for performing frame sampling processing on the target video according to the frame sampling frequency to obtain a sampling frame, and sending the sampling frame to the cloud server;

[0015] A model updating module is used to obtain model updating information determined by the cloud server according to the sampling frame, and update the first object recognition model.

[0016] In the third aspect, the present application also provides an environmental information collection device, which includes a shooting device, a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the shooting device is used to shoot a target location, and when the computer program is executed by the processor, the steps of the object recognition method for the environmental information collection device as described above are implemented.

[0017] In a fourth aspect, the present application also provides an object recognition system, which includes an environmental information collection device and a cloud server. The environmental information collection device is communicatively connected to the cloud server. The environmental information collection device is used to implement the steps of the object recognition method provided in the first aspect. The cloud server is used to receive a sampling frame sent by the environmental information collection device; train a second object recognition model according to the sampling frame to determine model update information; and send the model update information to the environmental information collection device.

[0018] In a fifth aspect, the present application also provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the object recognition method for an environmental information acquisition device as described above are implemented.

[0019] The present application provides an object recognition method, apparatus, device, system and storage medium for an environmental information acquisition device. The present application adjusts the frame sampling frequency according to the label change information corresponding to each frame of the target video to realize adaptive frame sampling in the environmental information acquisition device. The environmental information acquisition device reduces the data sent to the cloud server through adaptive frame sampling, reduces the bandwidth from the environmental information acquisition device to the cloud server and reduces the load on the cloud server; the environmental information acquisition device also sends the sampled sampled frames to the cloud server, so that the cloud server can determine the model update information of the first object recognition model in the environmental information acquisition device according to the sampled frames, so that the first object recognition model can learn from the cloud server, without deploying complex models in the environmental information acquisition device, reducing the computing load of the environmental information acquisition device, and enabling the first object recognition model to be updated in time, thereby improving the accuracy of object recognition when data drift occurs, and while improving the accuracy of object recognition, it can improve the efficiency of finding the video recording corresponding to the time of the car accident when a car accident occurs, thereby achieving the purpose of quickly identifying the responsible party and improving the claims efficiency of motor vehicle insurance cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of an object recognition method provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of the process steps of an object recognition method provided in another embodiment of the present application;

[0023] Figure 3 A scene diagram of an object recognition method provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an object recognition device provided by an embodiment of the present application;

[0025] Figure 5 A schematic block diagram of the structure of an environmental information collection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0028] The embodiments of the present application provide an object recognition method, device, environmental information collection device and storage medium. Among them, the object recognition method is applied to the environmental information collection device, which can be a video shooting device with certain computing capabilities, or a terminal device that communicates with the shooting device, such as a tablet computer, a desktop computer, etc. It should be noted that the environmental information collection device is connected to the cloud server in communication, and the cloud server can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0029] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0030] Please refer to Figure 1 , Figure 1 A flowchart of an object recognition method provided in an embodiment of the present application.

[0031] like Figure 1 As shown, the object recognition method includes steps S101 to S105.

[0032] Step S101: shoot the target location to obtain a target video.

[0033] Exemplarily, a shooting device is provided on the environmental information collection device, or the environmental information collection device is communicatively connected with the shooting device, so that the environmental information collection device can shoot the target position through the shooting device to obtain the target video.

[0034] Exemplarily, the target location may be an intersection, so that vehicles passing through the intersection can be photographed to obtain a traffic video, so that the environmental information acquisition device can utilize a neural network to identify vehicles based on the traffic video, thereby realizing traffic detection; the environmental information acquisition device may also be a shooting device provided on the target vehicle, so that the target vehicle can identify the road conditions of the target location of the road ahead according to the environmental information acquisition device, so as to realize functions such as automatic driving according to the identified road conditions.

[0035] Step S102: identifying a target object in the target video based on the trained first object recognition model, and determining label change information of video frames in the target video according to the target object.

[0036] Exemplarily, the environmental information acquisition device is deployed with a first object recognition model, wherein the first object recognition model is a lightweight model, and the trained first object recognition model can recognize the target object in the target video.

[0037] Specifically, in a resource-constrained environmental information acquisition device, in order to ensure smooth operation of object recognition in the video, the first object recognition model can be a YOLOv4 model with Resnet18 as the backbone network.

[0038] Exemplarily, the environmental information acquisition device can identify the target object in the target video based on the first object recognition model, and while using the first object recognition model to identify the target object, determine the label change information corresponding to the video frame in the target video. It should be understood that the label change information corresponding to the video frame is used to indicate the change of the target object in the target video.

[0039] In some embodiments, the target object in the target video is identified based on the trained first object recognition model, and the label change information of the video frame in the target video is determined according to the target object, including: identifying the target object of the video frame in the target video based on the first object recognition model; determining the label information corresponding to the video frame according to the target object of the video frame; and calculating the label change information of the video frame according to the label information of the video frame based on a preset target loss function.

[0040] Exemplarily, based on the trained first object recognition model, the target object in the target video is identified. Specifically, the target video is split into multiple video frames, and the target object is identified in each video frame. It can be understood that when the target object is recognized for each video frame, the corresponding label information will be output. Therefore, the label change information of each video frame can be determined based on the label information corresponding to each video frame.

[0041] During the specific implementation process, the first object recognition model determines the label information corresponding to the current video frame when performing target object recognition on the current video frame, and determines the label change information of the current video frame based on the label information corresponding to the current video frame and the label information corresponding to the video frame at the previous moment through a preset target loss function.

[0042] In another implementation process, the label change information of the current video frame is determined according to the label information corresponding to the current video frame and the label information corresponding to the initial video frame through a preset target loss function.

[0043] It should be understood that the video frame corresponding to the start time of the target video is taken as the initial video frame. It can be seen that the target video does not have a video frame at the previous moment of the initial video frame, so the initial video frame does not have corresponding label change information.

[0044] Exemplarily, the objective loss function is a CIOU loss function.

[0045] It should be noted that the label change information is used to indicate the rate of change of each video frame over time. The label change information can be determined based on the label information corresponding to adjacent video frames, or based on the label information of the current video frame and the label information of the initial video frame. Compared with the initial video frame, the label information is usually taken and determined in a smaller space in the image (i.e., the detection object class, such as a vehicle). It can be seen that the label change information is the loss of the first object recognition model's prediction of the label information of the current video frame and the label information of the frame at the previous moment, or the label information of the current video frame and the label information of the initial video frame. Therefore, the smaller the change rate corresponding to the label change information, the more similar the label information of the current video frame is to the label information of the video frame at the previous moment, that is, slowly changing or static scenes tend to make the change rate corresponding to the label change information obtain a lower score to determine the environmental scene changes of the target video.

[0046] Step S103: adjusting the frame sampling frequency of the environmental information acquisition device according to the tag change information.

[0047] Exemplarily, the frame sampling frequency of the target video by the environmental information acquisition device is adjusted according to the determined label change information, so as to dynamically adjust the frame sampling frequency of the environmental information acquisition device according to the degree and speed of scene changes in the video, thereby reducing the uplink (environmental information acquisition device to the cloud server) bandwidth and server load.

[0048] In some embodiments, adjusting the frame sampling frequency of the environmental information acquisition device according to the label change information includes: after the current time window passes, adjusting the frame sampling frequency of the environmental information acquisition device according to the label change information corresponding to the frame in the current time window.

[0049] Exemplarily, after a preset time window has passed, the frame sampling frequency of the environment information acquisition device is adjusted according to the label change information corresponding to the video frame in the time window, so as to achieve regular update of the frame sampling frequency of the environment information acquisition device.

[0050] Specifically, the time window is 10 seconds, that is, the corresponding video frames within the 10 seconds are determined in the target video, and the label change information corresponding to these video frames is determined, so that the frame sampling frequency of the environmental information acquisition device can be adjusted according to the label change information corresponding to these video frames.

[0051] In some embodiments, after the current time window passes, the frame sampling frequency of the environmental information acquisition device is adjusted according to the label change information corresponding to the video frame in the current time window, including: determining the video frame in the current time window as the target frame; based on a preset frame sampling calculation formula, determining the target frame sampling frequency according to the frame sampling frequency of the historical time window, the label change information of the target frame, the preset step parameter and the preset label change threshold; adjusting the frame sampling frequency of the environmental information acquisition device to the target frame sampling frequency.

[0052] Exemplarily, a video frame appearing in the current time window is determined as a target frame, and label change information corresponding to the target frame is determined, so that the target frame sampling frequency can be determined based on a preset frame sampling calculation formula according to the frame sampling frequency of the historical time window, label change information of the target frame, a preset step size parameter, and a preset label change threshold. The historical time window may be a previous time window.

[0053] Specifically, the preset frame sampling calculation formula is as follows:

[0054]

[0055] Among them, r t+1 is the target frame sampling frequency, r t Used to indicate the frame sampling frequency of the previous time window, η r Used to indicate the preset step size parameter. It is used to indicate the average label change information of all frames in the current time window, φ target Used to indicate the preset label change threshold; the range of the target frame sampling frequency is [r min , r max ].

[0056] It should be understood that the average label change information of all video frames in the current time window is determined according to the label change information corresponding to each video frame in the current time window. In one implementation, r min =0.1fps, r max =1fps.

[0057] Step S104: performing frame sampling processing on the target video according to the adjusted frame sampling frequency to obtain sampling frames, and sending the sampling frames to the cloud server.

[0058] Exemplarily, the target video is subjected to frame sampling processing according to the target frame sampling frequency to obtain the collected frames, and the collected frames are sent to the cloud server. It should be understood that the target video is subjected to frame sampling processing in each time window. Since the target sampling frequency corresponding to each time window may be different, the number of frames collected in each time window may also be different. For example, the video scene in the previous time window changes slowly, the target sampling frequency corresponding to the time window is low, and the number of collected frames is 560 frames, while the video scene in the current time window changes quickly, the target sampling frequency corresponding to the current time window is high, and the number of collected frames is 1600 frames, so that more frames can be collected for videos with constantly changing scenes, while fewer frames can be collected for videos with small scene changes, thereby improving the resource utilization of the cloud server.

[0059] Step S105: receiving model update information determined according to the sampling frame and fed back by the cloud server, and updating the first object recognition model.

[0060] It can be understood that the environmental information acquisition device can identify the target object in the target video based on the updated first object recognition model to improve the accuracy of target object recognition.

[0061] Exemplarily, the environmental information acquisition device sends the acquisition frame to the cloud server, so that the cloud server determines the model update information of the first object recognition model based on the acquisition frame. The environmental information acquisition device receives the model update information fed back by the cloud server, and completes the update of the first object recognition model according to the model update information, so that the first object recognition model can adapt to the changes in the scene and improve the accuracy of object recognition, so as to solve the problem that the environmental information acquisition device deploys a lightweight object recognition model, and when data drift occurs, such as scene changes, lighting changes, building size and color changes, the object recognition model deployed in the environmental information acquisition device can still maintain a high object recognition accuracy, thereby improving the performance of the object recognition model in real-time video streams.

[0062] The object recognition method provided in the above embodiment adjusts the frame sampling frequency according to the label change information corresponding to each frame of the target video to realize adaptive frame sampling in the environmental information acquisition device. The environmental information acquisition device reduces the data sent to the cloud server through adaptive frame sampling, reduces the bandwidth from the environmental information acquisition device to the cloud server and reduces the load on the cloud server; the environmental information acquisition device also sends the sampled sampled frames to the cloud server, so that the cloud server can determine the model update information of the first object recognition model in the environmental information acquisition device according to the sampled frames, so that the first object recognition model can learn from the cloud server, without deploying complex models in the environmental information acquisition device, reducing the computing load of the environmental information acquisition device, and enabling the first object recognition model to be updated in time using the model update information, thereby improving the object recognition accuracy of the first object recognition model when data drift occurs.

[0063] In some usage scenarios of the present application, when a car accident occurs on the road and video recording is required to collect evidence, so as to determine the responsible party and make corresponding car insurance claims, the method provided by the present application can automatically identify the vehicle involved in the accident in the monitoring video recording, and intercept the video recording within the time period of the accident through the identification result, so that the insurance personnel, traffic police and / or smart terminal can determine the responsible party based on the intercepted video recording. The method provided by the present application avoids the situation where vehicle identification errors occur during data drift, such as weather changes or environmental changes. For example, the vehicle involved in the accident is incorrectly identified, resulting in no accident in the intercepted video, and manual review and verification are required. The object recognition method provided by the present application can ensure relatively accurate object recognition when data drifts, realize video reasoning, and improve the accuracy of vehicle identification. Therefore, when a car accident occurs and a claim is required, there is no need to manually search the video to determine the responsible party of the accident, thereby improving the efficiency and convenience of car insurance claims.

[0064] See also Figure 2 , Figure 2 A schematic diagram of the process steps of another object recognition method provided in this application.

[0065] The present application also provides an object recognition system, which includes an environment acquisition device and a cloud server, wherein the environment acquisition device is communicatively connected to the cloud server; the environment information acquisition device is used to implement the object recognition method as described in any of the above embodiments, and the cloud server is used to implement Figure 2 Steps shown.

[0066] like Figure 2 As shown, the steps implemented by the cloud server include steps S201 to S203.

[0067] Step S201: Receive a sampling frame sent by the environmental information collection device.

[0068] Exemplarily, a sampling frame sent by the environmental information collection device is received to determine model update information for a first object recognition model in the environmental information collection device according to the sampling frame.

[0069] Specifically, the sampling frames sent by the environmental information collection device in each time window are obtained, and the sampling frames are stored.

[0070] Step S202: Train the second object recognition model according to the sampling frame to determine model update information.

[0071] Exemplarily, when the sampling frame sent by the environmental information acquisition device meets the model training conditions, the second object recognition model is trained according to the sampling frame to determine the model update information.

[0072] It should be understood that the second object recognition model and the first object recognition model are the same model to achieve synchronous training and updating, thereby ensuring the training accuracy of the first object recognition model.

[0073] See also Figure 3 , Figure 3 A scene diagram of an object recognition method provided in an embodiment of the present application.

[0074] In some embodiments, the cloud server is used to train the second object recognition model based on the sampling frames and determine the model update information, including: when the number of the sampling frames is greater than or equal to a preset frame number threshold, the cloud server determines the label information of the sampling frames based on the second object recognition model, and determines the model update information based on the labeled sampling frames and the label information.

[0075] Exemplarily, the cloud server obtains the sampling frames sent by the environmental information acquisition device and stores the sampling frames. When the number of the obtained sampling frames is greater than or equal to a preset number threshold, the cloud server trains the second object recognition model to determine the model update information.

[0076] In a specific implementation process, after the cloud server completes training using the sampling frames, the stored sampling frames may be deleted to reduce the load of the cloud server.

[0077] For example, when the preset frame number threshold is 2000 frames, if the cloud server obtains 560 frames and has completed training once before, the number of frames obtained is less than the preset frame number threshold, and the model is not trained. The cloud server continues to obtain the sampling frames sent by the environmental information acquisition device until the cumulative number of sampling frames obtained is greater than or equal to 2000 frames. At this time, the second object recognition model is trained to determine that the model has more information.

[0078] By setting a preset frame number threshold in the cloud server and adjusting the frame sampling frequency in the environmental information acquisition device to adjust the number of frames sent to the cloud server, the model can be frequently retrained when the video scene is constantly changing. When the video scene changes very little, the model can be retrained after a long period of time, making the system composed of the environmental information acquisition device and the cloud server more robust.

[0079] In a specific implementation process, the sampling frames are manually annotated to obtain annotated sampling frames.

[0080] In some embodiments, the cloud server is further used to annotate the sampling frames based on the trained teacher model to obtain annotated sampling frames.

[0081] Exemplarily, when the number of sampled frames is greater than or equal to a preset frame number threshold, the cloud server annotates the sampled frames based on the pre-deployed teacher model to obtain annotated sampled frames, and determines the model update information according to the annotated sampled frames and the label information corresponding to the sampled frames. The sampled frames are annotated by the teacher model without manual annotation, which improves the updating efficiency of the model.

[0082] It should be noted that the teacher model has more layers and parameters than the first object recognition model and / or the second object recognition model, and also has higher accuracy. Specifically, the teacher model is a Mask R-CNN with ResNeXt101 as the backbone network. It should be understood that when the first object recognition model deployed in the environmental information acquisition device can only remember a limited number of objects and scene changes, the first object recognition model is continuously trained and updated through the sampling frames obtained from the environmental information acquisition device by the cloud server, so that the first object recognition model can be continuously updated with the latest data and the changing object distribution and scene, thereby improving the object recognition accuracy of the first object recognition model when data drifts.

[0083] Step S203: Send the model update information to the environmental information collection device.

[0084] Exemplarily, the model update information is sent to the environment information collection device, so that the environment information collection device updates the first object recognition model according to the model update information, thereby improving the object recognition accuracy of the first object recognition model.

[0085] See also Figure 4 , Figure 4 It is a schematic diagram of an object recognition device provided in an embodiment of the present application. The object recognition device can be configured in an environmental information collection device to execute the aforementioned object recognition method for the environmental information collection device.

[0086] like Figure 4 As shown, the object recognition device includes: a video shooting module 110, an object recognition module 120, a frequency adjustment module 130, a sampling processing module 140, and a model updating module 150.

[0087] The video shooting module 110 shoots a video of the target location to obtain a target video.

[0088] The object recognition module 120 is used to recognize the target object in the target video based on the trained first object recognition model, and determine the label change information of the video frame in the target video according to the target object.

[0089] The frequency adjustment module 130 is used to adjust the frame sampling frequency of the environmental information collection device according to the tag change information.

[0090] The sampling processing module 140 is used to perform frame sampling processing on the target video according to the frame sampling frequency to obtain sampling frames, and send the sampling frames to the cloud server.

[0091] The model updating module 150 is used to obtain the model updating information determined by the cloud server according to the sampling frame, and update the first object recognition model.

[0092] Exemplarily, the object recognition module 120 includes a target object recognition submodule, a label information determination submodule, and a label change information determination submodule.

[0093] The target object recognition submodule is used to recognize the target object in each video frame of the target video based on the trained first object recognition model.

[0094] The label information determination submodule is used to determine the label information corresponding to each of the video frames according to the target object in each of the video frames.

[0095] The label change information determination submodule is used to determine the label change information of each video frame in the target video except the initial video frame based on a preset target loss function and according to the label information corresponding to each adjacent video frame.

[0096] Exemplarily, the frequency adjustment module 130 is further used to adjust the frame sampling frequency of the environmental information acquisition device according to the label change information corresponding to the video frame in the current time window after the current time window has passed.

[0097] Exemplarily, the frequency adjustment module 130 includes a target frame determination submodule, a target frame sampling frequency determination submodule, and a frame sampling frequency update submodule.

[0098] The target frame determination submodule is used to determine the video frame in the current time window as the target frame.

[0099] The target frame sampling frequency determination submodule is used to determine the target frame sampling frequency based on the preset frame sampling frequency calculation formula, the frame sampling frequency of the historical time window, the label change information of the target frame, the preset step size parameter and the preset label change threshold.

[0100] The frame adopts a frequency updating submodule, which is used to adjust the frame sampling frequency of the environmental information acquisition device to the target frame sampling frequency.

[0101] See also Figure 5 , Figure 5 A schematic block diagram of the structure of an environmental information collection device provided in an embodiment of the present application.

[0102] like Figure 5 As shown, the environmental information acquisition device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a storage medium and an internal memory.

[0103] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any object recognition method.

[0104] The processor is used to provide computing and control capabilities to support the operation of the entire environmental information collection equipment.

[0105] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any object recognition method.

[0106] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will appreciate that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the environmental information collection device to which the scheme of the present application is applied. The specific environmental information collection device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0108] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0109] Shoot the target location to obtain the target video;

[0110] Based on the trained first object recognition model, identify the target object in the target video, and determine label change information of the video frame in the target video according to the target object;

[0111] Adjusting the frame sampling frequency of the environmental information acquisition device according to the tag change information;

[0112] Perform frame sampling processing on the target video according to the frame sampling frequency to obtain a sampling frame, and send the sampling frame to the cloud server;

[0113] Receive model update information determined according to the sampling frame and fed back by the cloud server, and update the first object recognition model.

[0114] In one embodiment, when the processor recognizes the target object in the target video based on the trained first object recognition model and determines the label change information of each frame in the target video according to the target object, it is used to implement:

[0115] Based on the first object recognition model, identifying a target object in a video frame of the target video;

[0116] Determine label information corresponding to the video frame according to the target object of the video frame;

[0117] Based on the preset target loss function, the label change information of the video frame is calculated according to the label information corresponding to the video frame.

[0118] In one embodiment, when the processor adjusts the frame sampling frequency of the environmental information acquisition device according to the tag change information, it is used to implement:

[0119] After the current time window has passed, the frame sampling frequency of the environmental information acquisition device is adjusted according to the label change information corresponding to the frames in the current time window.

[0120] In one embodiment, when the processor adjusts the frame sampling frequency of the environmental information acquisition device according to the label change information corresponding to the frame in the current time window after the current time window has passed, it is used to implement:

[0121] Determine the frame within the current time window as the target frame;

[0122] Based on a preset frame sampling frequency calculation formula, the target frame sampling frequency is determined according to the frame sampling frequency of the historical time window, the label change information of the target frame, the preset step size parameter and the preset label change threshold;

[0123] The frame sampling frequency of the environmental information acquisition device is adjusted to the target frame sampling frequency.

[0124] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of object recognition described above can refer to the corresponding process in the aforementioned object recognition method embodiment for environmental information acquisition equipment, and will not be repeated here.

[0125] An embodiment of the present application further provides a storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the object recognition method of the present application.

[0126] The computer-readable storage medium may be an internal storage unit of the environmental information collection device described in the above embodiments, such as a hard disk or memory of the environmental information collection device. The computer-readable storage medium may also be an external storage device of the environmental information collection device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the environmental information collection device.

[0127] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0128] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0129] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An object recognition method, characterized in that: For an environmental information collection device, the environmental information collection device is communicatively connected to a cloud server, and the method includes: Shoot the target location to obtain the target video; Based on the trained first object recognition model, identify the target object in the target video, and determine label change information of the video frame in the target video according to the target object; Adjusting the frame sampling frequency of the environmental information acquisition device according to the tag change information; Perform frame sampling processing on the target video according to the frame sampling frequency to obtain a sampling frame, and send the sampling frame to the cloud server; Receive model update information determined according to the sampling frame and fed back by the cloud server, and update the first object recognition model.

2. The object recognition method according to claim 1, characterized in that: The step of identifying the target object in the target video based on the trained first object recognition model, and determining label change information of the video frame in the target video according to the target object, includes: Based on the first object recognition model, identifying a target object in a video frame of the target video; Determining label information corresponding to the video frame according to the target object of the video frame; Based on a preset target loss function, label change information of the video frame is calculated according to label information corresponding to the video frame.

3. The object recognition method according to claim 2, characterized in that: The step of adjusting the frame sampling frequency of the environmental information acquisition device according to the tag change information includes: After the current time window has passed, the frame sampling frequency of the environmental information acquisition device is adjusted according to the label change information corresponding to the video frame in the current time window.

4. The object recognition method according to claim 3, characterized in that: After the current time window has passed, adjusting the frame sampling frequency of the environmental information acquisition device according to the label change information corresponding to the video frame in the current time window includes: Determine the video frame within the current time window as the target frame; Based on a preset frame sampling frequency calculation formula, the target frame sampling frequency is determined according to the frame sampling frequency of the historical time window, the label change information of the target frame, the preset step size parameter and the preset label change threshold; The frame sampling frequency of the environmental information acquisition device is adjusted to the target frame sampling frequency.

5. An object recognition device, characterized in that: The object recognition device comprises: A video shooting module is used to shoot a video of the target location to obtain a target video; An object recognition module, configured to recognize a target object in the target video based on a first object recognition model, and determine label change information of each frame in the target video; A frequency adjustment module, used to adjust the frame sampling frequency of the environmental information collection device according to the tag change information; A sampling processing module, used for performing frame sampling processing on the target video according to the adjusted frame sampling frequency, and sending the collected sampling frames to the cloud server; A model updating module is used to obtain model updating information determined by the cloud server according to the sampling frame, and to update the first object recognition model.

6. An environmental information collection device, characterized in that: The environmental information acquisition device includes a shooting device, a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the shooting device is used to shoot a target location, and when the computer program is executed by the processor, the steps of the object recognition method as described in any one of claims 1 to 4 are implemented.

7. An object recognition system, characterized in that: The object recognition system comprises an environment information collection device and a cloud server, wherein the environment information collection device is communicatively connected to the cloud server, the environment information collection device is used to implement the steps of the object recognition method according to any one of claims 1 to 4, and the cloud server is used to receive the sampling frame sent by the environment information collection device; The second object recognition model is trained according to the sampling frame to determine model update information; and the model update information is sent to the environmental information acquisition device.

8. The object recognition system according to claim 7, characterized in that: The cloud server is used to train the second object recognition model according to the sampling frame to determine the model update information, including: When the number of the sampling frames is greater than or equal to a preset frame number threshold, the cloud server determines the label information of the sampling frames based on the second object recognition model, and determines the model update information according to the labeled sampling frames and the label information.

9. The object recognition system according to claim 8, characterized in that: The cloud server is also used to perform labeling processing on the sampling frames based on the trained teacher model to obtain labeled sampling frames.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein when the computer program is executed by the processor, the steps of the object recognition method according to any one of claims 1 to 4 are implemented.

Images