Image recognition method, electronic equipment and storage medium

Abstract

The invention discloses an image recognition method, electronic equipment and a storage medium. The method relates to the field of target tracking, and comprises the following steps: obtaining video frames which are obtained by shooting a real scene through a dome camera according to different visual angles; tracking a target object in the video frame to obtain a first track of the target object; a target key view angle matched with the video frame is determined from the multiple key view angles, different key view angles correspond to different recognition algorithms, and the different recognition algorithms are used for recognizing object behaviors in different areas; and identifying the target object based on the first track by using a target identification algorithm corresponding to the target key view angle to obtain a target identification result of the target object, the target identification result being used for representing whether the target object executes a target behavior corresponding to the target identification algorithm. The technical problems of low intelligent degree and low analysis efficiency of video analysis of the dome camera caused by manual processing of the video acquired by the dome camera in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of target tracking, and in particular to an image recognition method, an electronic device and a storage medium. Background Art

[0002] With the development of society and the progress of science and technology, the continuous development of computer technology has made it possible to manage and secure public places. For example, traffic systems, banking tasks and other scenes can be monitored by ball cameras. However, when analyzing the videos acquired by ball cameras, manual processing of the videos is required. For example, the videos acquired by ball cameras need to be divided into regions manually. However, the videos acquired by ball cameras are massive. When processed manually, it takes a long time, and inaccurate processing will occur when manually processing massive videos. Therefore, the intelligence level of ball camera video analysis is low and the analysis efficiency is low.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide an image recognition method, an electronic device, and a storage medium to at least solve the technical problem in the related art of low intelligence and low analysis efficiency of the video analyzed by the ball camera due to manual processing of the video acquired by the ball camera.

[0005] According to one aspect of an embodiment of the present application, an image recognition method is provided, including: acquiring a video frame, wherein the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; tracking a target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas; using a target recognition algorithm corresponding to the target key viewing angle, based on the first trajectory, the target object is recognized to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0006] According to another aspect of an embodiment of the present application, an image recognition method is also provided, including: acquiring a video frame, wherein the video frame is a video frame obtained by shooting a traffic scene with a ball camera according to different viewing angles; tracking the traffic target in the video frame to obtain a first trajectory of the traffic target; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different event analysis algorithms, and different event analysis algorithms are used to analyze traffic events in different traffic areas; using the event analysis algorithm corresponding to the target key viewing angle, analyzing the traffic events in the traffic scene based on the first trajectory to obtain a traffic event analysis result, wherein the traffic event analysis result is used to characterize whether a target traffic event corresponding to the target event analysis algorithm occurs in the traffic scene.

[0007] According to another aspect of an embodiment of the present application, an image recognition method is also provided, including: responding to an input instruction acting on an operation interface, displaying a video frame on the operation interface, wherein the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; responding to an image recognition instruction acting on the operation interface, displaying a target recognition result of a target object in the video frame on the operation interface, wherein the target recognition result is obtained by identifying the target object based on a first trajectory of the target object using a target recognition algorithm corresponding to a target key viewing angle, the first trajectory being obtained by tracking the target object in the video frame, the target key viewing angle being a key viewing angle that matches the video frame determined from multiple key viewing angles, different recognition algorithms corresponding to different key viewing angles, different recognition algorithms being used to recognize object behaviors in different areas, and the target recognition result being used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0008] According to another aspect of an embodiment of the present application, an image recognition method is also provided, including: displaying a video frame on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the video frame is obtained by shooting a real scene with a ball camera at different viewing angles; tracking a target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; using a target recognition algorithm corresponding to the target key viewing angle, identifying the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; and driving the VR device or AR device to render and display the target recognition result.

[0009] According to another aspect of an embodiment of the present application, an image recognition method is also provided, including: obtaining a video frame by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is a video frame, and the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; tracking the target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; using the target recognition algorithm corresponding to the target key viewing angle, identifying the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; outputting the target recognition result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target recognition result.

[0010] According to another aspect of an embodiment of the present application, an image recognition device is also provided, including: an acquisition module, used to acquire video frames, wherein the video frames are video frames obtained by shooting real scenes with a ball camera according to different viewing angles; a tracking module, used to track the target object in the video frame to obtain a first trajectory of the target object; a determination module, used to determine a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; an identification module, used to identify the target object based on the first trajectory using the target recognition algorithm corresponding to the target key viewing angle, and obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0011] According to another aspect of an embodiment of the present application, an image recognition device is also provided, including: an acquisition module, used to acquire video frames, wherein the video frames are video frames obtained by shooting traffic scenes with a ball camera according to different viewing angles; a tracking module, used to track traffic targets in the video frames to obtain a first trajectory of the traffic targets; a determination module, used to determine a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different event analysis algorithms, and different event analysis algorithms are used to analyze traffic events in different traffic areas; an analysis module, used to analyze traffic events in the traffic scene based on the first trajectory using the event analysis algorithm corresponding to the target key viewing angle, and obtain a traffic event analysis result, wherein the traffic event analysis result is used to characterize whether a target traffic event corresponding to the target event analysis algorithm occurs in the traffic scene.

[0012] According to another aspect of an embodiment of the present application, an image recognition device is also provided, including: a first display module, for responding to an input instruction acting on an operation interface, and displaying a video frame on the operation interface, wherein the video frame is a video frame obtained by shooting a real scene at different viewing angles using a ball camera; a second display module, for responding to an image recognition instruction acting on the operation interface, and displaying a target recognition result of a target object in the video frame on the operation interface, wherein the target recognition result is obtained by identifying the target object based on a first trajectory of the target object using a target recognition algorithm corresponding to a target key viewing angle, the first trajectory being obtained by tracking the target object in the video frame, the target key viewing angle being a key viewing angle that matches the video frame determined from multiple key viewing angles, different recognition algorithms corresponding to different key viewing angles, different recognition algorithms being used to recognize object behaviors in different areas, and the target recognition result being used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0013] According to another aspect of an embodiment of the present application, an image recognition device is also provided, including: a first display module, used to display video frames on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the video frames are obtained by shooting a real scene with a ball camera according to different viewing angles; a tracking module, used to track a target object in the video frame to obtain a first trajectory of the target object; a determination module, used to determine a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; an identification module, used to identify the target object based on the first trajectory using a target recognition algorithm corresponding to the target key viewing angle, to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; a second display module, used to drive the VR device or AR device to render and display the target recognition result.

[0014] According to another aspect of an embodiment of the present application, an image recognition device is also provided, including: an acquisition module, used to acquire a video frame by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is a video frame, and the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; a tracking module, used to track a target object in the video frame to obtain a first trajectory of the target object; a determination module, used to determine a target key viewing angle matching the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; an identification module, used to identify the target object based on the first trajectory using a target recognition algorithm corresponding to the target key viewing angle, and obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; an output module, used to output the target recognition result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target recognition result.

[0015] According to another aspect of an embodiment of the present application, an electronic device is further provided, including: a memory storing an executable program; and a processor for running the program, wherein any one of the above methods is executed when the program is running.

[0016] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any of the above methods.

[0017] In the embodiment of the present application, the purpose of identifying the behavior of the target object by the ball camera is achieved by acquiring a video frame; tracking the target object in the video frame to obtain a first trajectory of the target object; determining the target key perspective that matches the video frame from multiple key perspectives; using the target recognition algorithm corresponding to the target key perspective to identify the target object based on the first trajectory to obtain the target recognition result of the target object. It is easy to notice that since the ball camera can acquire video frames according to different perspectives, by determining the target key perspective that matches the video frame, and using the target recognition algorithm corresponding to the target key perspective to identify the target object based on the first trajectory, it is possible to automatically analyze the key angles that need to be identified based on any angle, and quickly and accurately obtain the target recognition result of the target object, thereby achieving the technical effect of improving the intelligence and efficiency of the ball camera's video analysis, and thus solving the technical problem of low intelligence and low efficiency of the ball camera's video analysis caused by manual processing of the video acquired by the ball camera in the related art.

[0018] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a schematic diagram of a hardware environment of a virtual reality device according to an image recognition method of an embodiment of the present application;

[0021] Figure 2 is a structural block diagram of a computing environment of an image recognition method according to an embodiment of the present application;

[0022] Figure 3 is a flow chart of an image recognition method according to Embodiment 1 of the present application;

[0023] Figure 4 is a flow chart of an image recognition method according to Embodiment 2 of the present application;

[0024] Figure 5 is a flow chart of an image recognition method according to Embodiment 3 of the present application;

[0025] Figure 6 is a schematic diagram of an optional operation interface according to Example 3 of the present application;

[0026] Figure 7 is a flow chart of an image recognition method according to Embodiment 4 of the present application;

[0027] Figure 8 is a flow chart of an image recognition method according to Embodiment 5 of the present application;

[0028] Fig. 9 is a schematic diagram of an image recognition device according to Embodiment 6 of the present application;

[0029] Fig.10 is a schematic diagram of an image recognition device according to Embodiment 7 of the present application;

[0030] Fig.11 is a schematic diagram of an image recognition device according to Embodiment 8 of the present application;

[0031] Fig.12 is a schematic diagram of an image recognition device according to Embodiment 9 of the present application;

[0032] Fig.13is a schematic diagram of an image recognition device according to Embodiment 10 of the present application;

[0033] Fig.14 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0037] Dome camera: A security monitoring camera with a pan / tilt head that can rotate and zoom at its own angle.

[0038] PTZ: Pan / Tilt / Zoom, the three directions of the ball camera's rotation are horizontal angle, vertical angle, and zoom.

[0039] Video event analysis: By analyzing the targets in the video, various types of events are captured, such as traffic violations (failure to give way to pedestrians, non-motor vehicles driving in the wrong direction, etc.).

[0040] Deep learning: deep learning, deep neural network technology.

[0041] Coil: In video event analysis, the area that needs to be analyzed, such as identifying illegally parked vehicles, it is necessary to specify a non-illegally parked area in the picture. Commonly used methods include using polygons to represent areas and line segments to represent detection lines.

[0042] SURF: Speed ​​Up Robust Features, accelerated robust features.

[0043] Example 1

[0044] According to an embodiment of the present application, an image recognition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Figure 1 FIG. 1 is a schematic diagram of a hardware environment of a virtual reality device according to an image recognition method of an embodiment of the present application. Figure 1 As shown, the virtual reality device 104 is connected to the terminal 106, and the terminal 106 is connected to the server 102 through a network. The above-mentioned virtual reality device 104 is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines, etc., and the above-mentioned terminal 104 is not limited to PCs, mobile phones, tablet computers, etc. The server 102 can be a server corresponding to a media file operator, and the above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.

[0046] Optionally, the virtual reality device 104 of this embodiment includes: a memory, a processor, and a transmission device. The memory is used to store an application, which can be used to execute: obtaining a video frame, wherein the video frame is a video frame obtained by shooting a real scene by a ball camera according to different viewing angles; tracking the target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to recognize the behavior of objects in different areas; using the target recognition algorithm corresponding to the target key viewing angle, based on the first trajectory, the target object is recognized to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object performs the target behavior corresponding to the target recognition algorithm. Thus, the technical problem of low intelligence and low analysis efficiency of the ball camera's video analysis caused by manual processing of the video obtained by the ball camera in the related art is solved, and the purpose of the ball camera being able to intelligently and efficiently analyze the video is achieved.

[0047] The terminal of this embodiment can be used to display video frames on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the video frames are obtained by shooting a real scene at different viewing angles using a ball camera; a target object in the video frame is tracked to obtain a first trajectory of the target object; a target key viewing angle matching the video frame is determined from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas; a target recognition algorithm corresponding to the target key viewing angle is used to recognize the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; and the target recognition result is sent to the virtual reality device 104, and the virtual reality device 104 displays the target recognition result at the target placement position after receiving the target recognition result.

[0048] Optionally, the virtual reality device 104 of this embodiment has an eye-tracking HMD (Head Mount Display) head display and an eye-tracking module that have the same functions as those in the above embodiment, that is, the screen in the HMD head display is used to display real-time images, and the eye-tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyeballs. The terminal of this embodiment obtains the user's position information and movement information in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space.

[0049] Figure 1 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the above-mentioned AR / VR device (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Figure 2 The block diagram shows the use of the above Figure 1 The AR / VR device (or mobile device) shown is an embodiment of a computing node in the computing environment 201. Figure 2 is a structural block diagram of a computing environment of an image recognition method according to an embodiment of the present application, such as Figure 2 As shown, computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ..., in the figure). Different computing nodes contain local processing and memory resources, and end users 202 can remotely run applications or store data in computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in computing environment 201, representing services "A", "D", "E" and "H" respectively.

[0050] The end user 202 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 202's provision and / or request can be provided to the entry gateway 230. The entry gateway 230 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 201).

[0051] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0052] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 2 As shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). Pods can include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0053] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201, and executing one or more functions of one service may require invoking one or more functions of another service. Figure 2 As shown, service "A" 220-1 receives a user request from end user 202 from ingress gateway 230, service "A" 220-1 may call service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to perform one or more functions.

[0054] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0055] Under the above operating environment, this application provides Figure 3 It should be noted that the image recognition method of this embodiment can be Figure 1 The illustrated embodiment is executed by a mobile terminal. Figure 3 is a flow chart of an image recognition method according to Embodiment 1 of the present application. Figure 3 As shown, the method may include the following steps:

[0056] Step S302, obtaining video frames, wherein the video frames are obtained by shooting a real scene with a ball camera at different viewing angles.

[0057] The above-mentioned video frame can be a frame of video obtained by extracting a video after the ball camera shoots a real scene at any angle. The video can be extracted at equal time intervals or at any time intervals. In this embodiment, the extraction time interval of the video frame is not specifically limited. The above-mentioned real scene can be a traffic scene, a bank task scene, an entertainment scene, etc., but is not limited thereto, and can be any scene that can be shot using a ball camera. The above-mentioned different viewing angles can be different rotation angles of the ball camera, or different zoom ratios of the ball camera at the same rotation angle, etc., but is not limited thereto.

[0058] In an optional embodiment, when automatic monitoring of a traffic scene is required, first, the ball camera can shoot the traffic scene at different rotation angles to obtain a video, and secondly, the video can be extracted at equal or unequal time intervals to obtain a video frame.

[0059] In another optional embodiment, when it is necessary to monitor and manage the traffic scene, first, the ball camera can shoot the traffic scene at the same rotation angle and different zoom ratios to obtain video, and secondly, the video can be extracted at equal or unequal time intervals to obtain video frames.

[0060] In another optional embodiment, when it is necessary to monitor and manage the bank mission scene, the ball camera can first shoot the bank mission scene at different rotation angles to obtain a video, and then extract the video at equal or unequal time intervals to obtain video frames.

[0061] Step S304: track the target object in the video frame to obtain a first trajectory of the target object.

[0062] The target object mentioned above may be an object to be identified in the video frame. For example, in a traffic scene, the target object may include but is not limited to pedestrians, motor vehicles, non-motor vehicles, obstacles, etc. The target object may be one or more. The specific number can be set by the user according to actual needs and is not limited in this embodiment. The first trajectory mentioned above may reflect the moving direction and distance of the target object, which may be expressed as T raj (i, t), but not limited to this. Where i is the number of the target object and t is the time.

[0063] In an optional embodiment, after the video frame is acquired, the target object can be tracked by a target detection and tracking algorithm to obtain a first trajectory T of the target object. raj (i,t).

[0064] It should be noted that target detection and tracking detects and tracks subjects in video frames, such as motor vehicles, non-motor vehicles, and pedestrians, to form a first track of the target object. The target detection and tracking algorithm may be any one or more target detection and tracking algorithms or subsystems in the relevant technology, and is not specifically limited in this embodiment.

[0065] Step S306, determining a target key perspective that matches the video frame from the multiple key perspectives, wherein different key perspectives correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas.

[0066] The above-mentioned multiple key perspectives can be perspectives set by the user to identify the behavior of the target object. Optionally, whether the current perspective of the ball camera is a key perspective can be determined by whether the ball camera stays for more than a preset time. The preset time here can be 5 seconds, 10 seconds, 15 seconds, etc., but is not limited to this.

[0067] In an optional embodiment, the key viewing angle of the ball camera can be set in advance by the user, or the key viewing angle can be determined based on the dwell time of the ball camera during the process of behavior recognition through the ball camera. The above-mentioned different areas can be areas contained in the video image captured by the ball camera at the key viewing angle, that is, areas in the real scene where the behavior of the target object is expected to be recognized. Since the ball camera can often capture at least one area in the real scene at one viewing angle, the different areas corresponding to different key viewing angles can be one or more. The above-mentioned recognition algorithm can be an algorithm for recognizing the behavior of the target object in different areas. Optionally, since the target object in the same area may perform different behaviors, the recognition algorithms corresponding to different areas can also be one or more. The above-mentioned object behavior can be a behavior monitored in a real scene. For example, in a traffic scene, the object behavior can be the behavior of a motor vehicle not giving way to pedestrians or the behavior of a pedestrian not crossing the road on a zebra crossing, but is not limited thereto.

[0068] In another optional embodiment, the target key perspective that matches the video frame can be determined from multiple key perspectives through an image matching algorithm. Specifically, the video frame can be matched with key images corresponding to different key perspectives, and a key image with a high similarity to the video frame can be found from multiple key images, so that the key perspective corresponding to the key image with a high similarity is the target key perspective. Through the above scheme, any perspective of the ball camera can be matched to the key perspective to ensure the purpose of subsequent behavior recognition.

[0069] It should be noted that the above-mentioned image matching algorithm can be any one or more mature image matching algorithms in the prior art, and the user can set it according to actual needs, which is not limited in this embodiment.

[0070] Step S308, using the target recognition algorithm corresponding to the target key perspective, the target object is recognized based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0071] The above-mentioned target recognition algorithm can be a recognition algorithm corresponding to the target key perspective, which can be a single algorithm or a set of multiple algorithms. The above-mentioned target behavior can be a behavior that the user pre-sets and hopes to monitor in a real scene. For example, in a traffic scene, when the target recognition object is a motor vehicle, the target behavior can be a motor vehicle not giving way to pedestrians, or a motor vehicle driving in a bus lane, but not limited to this; when the target recognition object is a pedestrian, the target behavior can be a pedestrian not crossing the road on a zebra crossing, or a pedestrian walking on a motor vehicle lane, but not limited to this.

[0072] In an optional embodiment, after the target key perspective is determined, the first trajectory can be identified by the target recognition algorithm corresponding to the target key perspective to determine whether the behavior performed by the target object is the target behavior, thereby obtaining the target recognition result of the target object. i Determine the target recognition algorithm F i , assuming that the target recognition algorithm F i It is used to identify pedestrians who do not cross the road at the zebra crossing, and then can be based on F i and the first trajectory T raj (i, t) recognizes the behavior of pedestrians (i.e., target objects) and obtains the target recognition result of whether the pedestrians cross the road on the zebra crossing.

[0073] For example, taking the traffic scene as an example for detailed explanation, the ball camera can be installed in different positions of the traffic scene. Through operations such as rotating and zooming the ball camera, the traffic scene can be shot from different perspectives to obtain videos. The ball camera then sends the shot video to the background processing center, which extracts the video shot by the ball camera to obtain video frames. Secondly, the traffic targets in the video frames can be tracked through a general target detection and tracking algorithm to obtain a first trajectory corresponding to the traffic target. Then, the target key perspective can be determined from multiple key perspectives through an image matching algorithm, and the behavior of the target object can be identified through the target recognition algorithm corresponding to the target key perspective and the first trajectory to obtain a target recognition result, that is, the purpose of analyzing traffic events in traffic scenes is achieved. For example, in order to monitor whether motor vehicles give way to pedestrians at a crosswalk, a key perspective KA can be pre-set and a corresponding recognition algorithm F can be set. The recognition algorithm F can identify whether the motor vehicle gives way to pedestrians. Therefore, after the ball camera captures a video frame of any perspective, the motor vehicle in the video frame can be tracked first to obtain a first trajectory of the motor vehicle. If the target key perspective that matches the video frame is the above-mentioned pre-set key perspective KA, the above-mentioned recognition algorithm F can be used to process the first trajectory to achieve the purpose of identifying the behavior of the motor vehicle, thereby determining whether the motor vehicle in the video frame fails to give way to pedestrians, thereby achieving the purpose of analyzing and monitoring traffic incidents of motor vehicle failure to give way in traffic scenes.

[0074] In the embodiment of the present application, the purpose of identifying the behavior of the target object by the ball camera is achieved by acquiring a video frame; tracking the target object in the video frame to obtain a first trajectory of the target object; determining the target key perspective that matches the video frame from multiple key perspectives; using the target recognition algorithm corresponding to the target key perspective to identify the target object based on the first trajectory to obtain the target recognition result of the target object. It is easy to notice that since the ball camera can acquire video frames according to different perspectives, by determining the target key perspective that matches the video frame, and using the target recognition algorithm corresponding to the target key perspective to identify the target object based on the first trajectory, it is possible to automatically analyze the key angles that need to be identified based on any angle, and quickly and accurately obtain the target recognition result of the target object, thereby achieving the technical effect of improving the intelligence and efficiency of the ball camera's video analysis, and thus solving the technical problem of low intelligence and low efficiency of the ball camera's video analysis caused by manual processing of the video acquired by the ball camera in the related art.

[0075] In the above embodiment of the present application, a target key perspective that matches a video frame is determined from multiple key perspectives, including: performing feature extraction on key images corresponding to the multiple key perspectives to obtain multiple first pixel features, and performing feature extraction on the video frame to obtain second pixel features, the multiple first pixel features corresponding one-to-one to the multiple key perspectives; obtaining the distances between the multiple first pixel features and the second pixel features to obtain multiple distances; and determining that the key perspective corresponding to the minimum distance among the multiple distances is the target key perspective.

[0076] The above-mentioned key image can be a video frame obtained by the ball camera shooting the real scene according to the key perspective. Optionally, in order to ensure the accuracy of the key perspective matching, multiple video frames obtained by the ball camera shooting according to the key perspective for a period of time can be obtained, and then the multiple video frames are averaged to obtain the key image.

[0077] In an optional embodiment, first, point features of multiple key images can be extracted through an image matching algorithm to obtain multiple first pixel features {Feat_i} (for example, they can be SURF point features, but not limited to this, and can also be deep learning point features), and point features of video frames can be extracted through an image matching algorithm to obtain second pixel features Feat0, and then the distances between the multiple first pixel features and the second pixel features can be obtained to obtain multiple distances {dist_i}, and finally, the key viewing angle corresponding to the minimum distance dist_m among the multiple distances can be determined as the target key viewing angle.

[0078] In the above embodiment of the present application, determining the key perspective corresponding to the minimum distance among multiple distances as the target key perspective includes: determining whether the minimum distance is greater than a preset threshold; when the minimum distance is less than or equal to the preset threshold, determining the key perspective corresponding to the minimum distance as the target key perspective.

[0079] It should be noted that if the viewing angle corresponding to the video frame is different from multiple key viewing angles, resulting in a relatively large distance between the second pixel feature and multiple first pixel features, then if the key viewing angle corresponding to the minimum distance is directly used as the target key viewing angle, there will be a problem of misidentification. In order to avoid the above problem, a preset threshold can be set by the user in advance, and the preset threshold is used to distinguish whether the video frame is similar to multiple key images. The specific value can be set by the user according to actual needs and is not limited in this embodiment.

[0080] In an optional embodiment, after the minimum distance is determined, the minimum distance may be first compared with a preset threshold Thres. When the minimum distance is less than or equal to the preset threshold, the key viewing angle corresponding to the minimum distance may be determined as the target key viewing angle.

[0081] In the above embodiment of the present application, when the minimum distance is greater than a preset threshold, the method also includes: determining whether the target perspective corresponding to the video frame is a key perspective based on the first trajectory; when the target perspective is a key perspective, determining the key image corresponding to the target perspective based on multiple video frames within a preset time period, wherein the preset time period is a time period before the current moment and has a preset length; constructing a recognition algorithm corresponding to the target perspective based on the key image corresponding to the target perspective; and storing the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective in correspondence.

[0082] The above-mentioned preset time period can be a time period set in advance by the user for determining the key image, wherein the preset time period T is located before the current moment t0 and has a preset length, wherein the preset length can be set by the user according to actual needs and is not limited in this embodiment.

[0083] In an optional embodiment, when it is determined that the minimum distance is greater than a preset threshold, it can be determined that there is no target key view matching the video frame among the multiple key views. Therefore, it can be determined whether the target view corresponding to the video frame is a key view based on the first trajectory of the video frame. For example, it can be determined whether the target object is a stationary object through the first trajectory. If the target objects captured in the video frame are all stationary objects, it can be determined that the target view is not a key view; if the target objects captured in the video are all moving objects, it can be determined that the target view is likely to be a key view. After determining that the target view is a key view, multiple video frames within a preset time period before the current moment can be obtained, and the multiple video frames can be averaged, weighted, and other operations are performed to obtain a key image K corresponding to the target view. Then, the recognition algorithm F corresponding to the target view can be constructed based on the key image K. For example, the recognition algorithm can be manually set by the user, or the corresponding recognition algorithm can be selected from a pre-set recognition algorithm set by recognizing the key image, but it is not limited to this. Finally, the target view, the key image corresponding to the target view, and the recognition algorithm corresponding to the target view can be stored correspondingly to facilitate subsequent behavior recognition.

[0084] In the above embodiment of the present application, based on the first trajectory, determining whether the target perspective corresponding to the video frame is a key perspective includes: determining whether the video frame is a fixed picture; when the video frame is a fixed picture, determining whether the fixed duration corresponding to the fixed picture is greater than a preset duration, wherein the fixed duration is used to characterize the duration of the time structure corresponding to the fixed picture; when the fixed duration is greater than the preset duration, determining the number of targets located in a preset area of ​​the video frame among the trajectory points included in the first trajectory; when the number of targets is greater than the preset number, determining that the target perspective is a key perspective; when the video frame is not a fixed picture, or the fixed duration is less than or equal to the preset duration, or the number of targets is less than or equal to the preset number, determining that the target perspective is not a key perspective.

[0085] The above-mentioned preset duration can be set by the user according to actual needs, which can be 10 seconds, but not limited to this, and can also be 8 seconds, 12 seconds, etc. The above-mentioned preset area can be an area located in the center of the video frame divided in advance by the user, wherein the length and width of the preset area can be set to half of the video frame, but not limited to this. The above-mentioned preset number can be the number of track points set in advance by the user for determining the key perspective, and the specific value can be set by the user according to actual needs. In this embodiment, it can be 100, but not limited to this, and can also be 120, etc.

[0086] In an optional embodiment, first, the point feature matching of the current video frame and the previous frame of the video frame can be performed by an image matching algorithm (for example, it can be a SURF point feature, but not limited to this). For example, the pixel points of the current video frame can be matched with the pixel points of the previous frame of the current video frame. When the number of matched pixel points is greater than the preset number of pixel points, and the coordinate error of the matched pixel points is less than the preset error, the video frame can be determined to be a fixed picture. The specific value of the preset number of pixel points can be set by the user according to actual needs. In this embodiment, it can be 100, but not limited to this, and can also be 90, 110, etc. Similarly, the specific value of the preset error can also be set by the user according to actual needs. In this embodiment, it can be 3, but not limited to this, and can also be 1, 5, etc.

[0087] In another optional embodiment, after determining that the video frame is a fixed picture, it can be determined whether the fixed time length corresponding to the fixed picture is greater than 10 seconds (i.e., the preset time length). After determining that the fixed time length of the fixed picture is greater than 10 seconds, the first trajectory T can be further determined. raj Among the trajectory points included in (i, t), whether the number of targets in the preset area S is greater than the preset number; if it is determined that the number of targets of the trajectory points is greater than 100 (ie, the preset number), the target perspective can be determined to be the key perspective.

[0088] In another optional embodiment, when the video frame is not a fixed picture, or the fixed duration is less than 10 seconds, or the target number of trajectory points is less than 100, it is determined that the target perspective is not a key perspective, and the video frame does not need to be processed in any way and can be directly discarded.

[0089] In the above embodiment of the present application, determining the number of targets located in a preset area of ​​a video frame among the trajectory points included in the first trajectory includes: acquiring a second trajectory located in a preset time period in the first trajectory; determining a moving distance of the target object based on the second trajectory; eliminating target objects whose moving distance is less than the preset distance from the second trajectory to obtain a third trajectory; and determining the number of targets located in the preset area among the trajectory points included in the third trajectory.

[0090] The second track mentioned above may be the track of the first track within a preset time period, wherein the preset time period may be a period of time before the current moment. The preset distance mentioned above may be a distance set in advance by the user for screening stationary objects among target objects, and the specific value may be set by the user according to actual needs. In this embodiment, it may be 20 pixels, but it is not limited thereto, and may also be 15 pixels, 25 pixels, etc. The third track mentioned above may be the track of non-stationary objects within a preset time period.

[0091] In an optional embodiment, the first trajectory T can be obtained first. raj (i, t) The second trajectory T within the preset time period T raj_T (i, t), where t0-T≤t≤t0, t0 is the current moment corresponding to the video frame. Secondly, the distance difference between the first and last trajectory points in the first trajectory and the second trajectory can be obtained to obtain the moving distance of the target object. Then, the moving distance can be compared with the preset distance d, and the target objects with a moving distance less than d in the second trajectory are eliminated to obtain the third trajectory, that is, the trajectory of all non-stationary objects is obtained. Finally, the number of targets located in the preset area S among the trajectory points contained in the third trajectory can be determined.

[0092] It should be noted that when the moving distance is less than the preset distance d, it indicates that the target object corresponding to the moving distance is a stationary object. Since the behavior of the stationary object has not changed and does not need to be identified, the stationary object can be eliminated to reduce the amount of calculation.

[0093] In the above embodiment of the present application, determining the key image corresponding to the target viewing angle based on multiple video frames within a preset time period includes: obtaining an average value of the multiple video frames to obtain the key image corresponding to the target viewing angle.

[0094] In an optional embodiment, after determining that the target perspective is the key perspective, multiple video frames within a preset time period can be obtained, and then an averaging operation can be performed on the multiple video frames, and the average value of the multiple video frames is used as the key image corresponding to the target perspective.

[0095] In the above embodiments of the present application, a recognition algorithm corresponding to the target perspective is constructed based on the key image corresponding to the target perspective, including: performing image segmentation on the key image corresponding to the target perspective to obtain at least one semantic area; performing contour extraction on at least one semantic area to obtain a region contour corresponding to at least one semantic area; determining a recognition algorithm corresponding to at least one semantic area from multiple preset recognition algorithms; and summarizing the recognition algorithms corresponding to at least one semantic area to obtain a recognition algorithm corresponding to the target perspective.

[0096] In an optional embodiment, after determining the key image K corresponding to the target perspective, firstly, K can be segmented by an image segmentation algorithm to obtain at least one semantic region {Region_i}, where i is the number of the semantic region and can be 1, 2, 3, etc.; secondly, contour extraction can be performed on at least one semantic region to obtain a region contour {Contour_i} corresponding to at least one semantic region, where i is the number of the contour region and the region contour can be a line segment or a polygon, but is not limited thereto; then, the recognition algorithm Func_i corresponding to at least one semantic region can be determined based on a pre-set mapping table, where i represents the number of the recognition algorithm, and the mapping table can reflect the mapping relationship between different semantic regions and event analysis functions corresponding to different semantic regions, and at least one semantic region can be mapped to the corresponding event analysis functions. The corresponding recognition algorithm can be one or a set of multiple recognition algorithms; finally, the recognition algorithms corresponding to at least one semantic area can be summarized to obtain the recognition algorithm F corresponding to the target perspective. If different semantic areas correspond to one recognition algorithm, the recognition algorithm F corresponding to the target perspective can be expressed as F={[Contour_1, Func_1],[Contour_2,Func_2],···}; if different semantic areas correspond to multiple recognition algorithms, for example, taking different semantic areas corresponding to two recognition algorithms as an example, the recognition algorithm F corresponding to the target perspective can be expressed as F={[Contour_1,Func_11,Func_12],[Contour_2,Func_21,Func_22],···}, but is not limited to this.

[0097] It should be noted that the image segmentation algorithm may be any one or more image segmentation algorithms in the prior art, and is not specifically limited in this embodiment.

[0098] In another optional embodiment, after determining the key image K corresponding to the target perspective, firstly, K can be segmented by an image segmentation algorithm to obtain at least one semantic region {Region_i}, where i is the number of the semantic region and can be 1, 2, 3, etc.; secondly, contour extraction can be performed on at least one semantic region to obtain a region contour {Contour_i} corresponding to at least one semantic region, where i is the number of the contour region and the region contour can be a line segment or a polygon, but is not limited thereto; then the user can configure the recognition algorithm Func_i corresponding to at least one semantic region, where i represents the number of the recognition algorithm. For example, the user can select the recognition algorithm corresponding to different semantic regions from all the recognition algorithms, where at least one semantic region configured by the user is The recognition algorithm corresponding to the semantic area can be one or a set of multiple recognition algorithms. Finally, the recognition algorithms corresponding to at least one semantic area can be summarized to obtain the recognition algorithm F corresponding to the target perspective. If different semantic areas correspond to one recognition algorithm, the recognition algorithm F corresponding to the target perspective can be expressed as F={[Contour_1, Func_1],[Contour_2,Func_2],···}. If different semantic areas correspond to multiple recognition algorithms, for example, taking different semantic areas corresponding to two recognition algorithms as an example, the recognition algorithm F corresponding to the target perspective is F={[Contour_1,Func_11,Func_12],[Contour_2,Func_21,Func_22],···}, but is not limited to this.

[0099] In the above embodiments of the present application, the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective are stored correspondingly, including: determining whether the number of multiple key perspectives is greater than or equal to a preset number; when the number of multiple key perspectives is greater than or equal to the preset number, the historical key perspectives, the key images corresponding to the historical key perspectives, and the recognition algorithms corresponding to the historical key perspectives among the multiple key perspectives are eliminated; and the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective are stored correspondingly.

[0100] The preset number may be the maximum number of storage that can be set by the user in advance, and the specific value may be set by the user according to actual needs, which is not limited in this embodiment. The historical key perspective may be the key perspective that has been stored before the current moment.

[0101] In an optional embodiment, before the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective are correspondingly stored, it can be first determined whether the number of multiple key perspectives is greater than or equal to a preset number N. When it is determined that the preset number of key perspectives is greater than or equal to N, it indicates that the current storage capacity is insufficient and the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective cannot be stored. At this time, the historical key perspectives, the key images corresponding to the historical key perspectives, and the recognition algorithms corresponding to the historical key perspectives can be eliminated. For example, the historical key perspectives corresponding to the historical video frames with a longer interval time with the current video frames, the key images corresponding to the historical key perspectives, and the recognition algorithms corresponding to the historical key perspectives can be eliminated first, and then the target perspective, the key image corresponding to the target perspective, and the recognition algorithms corresponding to the target perspective can be correspondingly stored. That is, the target perspective, the key image corresponding to the target perspective, and the recognition algorithms corresponding to the target perspective can be put into Config{KA i} set, where Config is the target key perspective configuration, KA i ={K i ,F i}, where K is the key image corresponding to the target key perspective KA, and F is the recognition algorithm corresponding to the target key perspective.

[0102] In the above embodiment of the present application, a target recognition algorithm corresponding to a target key perspective is used to identify a target object based on a first trajectory to obtain a target recognition result of the target object, including: determining a homography matrix between a target key image corresponding to the target key perspective and a video frame; correcting a target area corresponding to the target recognition algorithm based on the homography matrix to obtain a corrected recognition algorithm; and identifying the target object based on the first trajectory using the corrected recognition algorithm to obtain a target recognition result.

[0103] In an optional embodiment, the unit matrix H between the target key image and the video frame can be first calculated. The homography matrix indicates the offset between the target key image and the video frame. Since the video frame is offset, the semantic area in the target key image has changed. Therefore, the matrix product of the unit matrix H and the target recognition algorithm can be obtained, and the contour of the target area is corrected to obtain a corrected recognition algorithm F', where F'={[H@Contour_1,Func_1],[H@Contour_2,Func_2],···}, where @ represents matrix multiplication. Finally, the target object can be identified based on the first trajectory through the corrected recognition algorithm, and the target recognition result can be obtained.

[0104] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0105] For example, taking the traffic scene as an example, this application combines three-dimensional analysis technology, image semantic segmentation technology, target detection and tracking technology and other technologies to implement a 360° adaptive event analysis system for the ball camera, which is truly plug-and-play and requires no manual operation.

[0106] The system of the present application includes multiple modules: a target detection and tracking module, a coil generation module, an arbitrary angle matching module, and a key angle selection and overall algorithm flow module.

[0107] The target detection and tracking module detects and tracks the subject (i.e., target object) in the video screen, such as a motor vehicle, a non-motor vehicle, or a pedestrian, to form a target trajectory. The specific target detection and tracking algorithm is not described in detail in this embodiment, and any target detection and tracking algorithm or subsystem can be applied. The target detection and tracking obtains the real-time trajectory data (i.e., the first trajectory) of all targets in the video screen, which is recorded as T raj (i, t), where i is the number of the target object and t is the time.

[0108] The coil generation module needs to input a key image K and output a coil configuration. The specific algorithm is briefly described as the following steps:

[0109] Step S1, segmenting K into different semantic regions {Region_i}, i=1, 2, 3...;

[0110] Step S2, performing image operations on different semantic regions {Region_i} to extract contours {Region_i} in the form of polygons or line segments;

[0111] Step S3, assigning different event analysis function configurations Func to different semantic areas such as motor vehicle lanes, non-motor vehicle lanes, pedestrian crossings, and intersections, such as failure to give way to pedestrians;

[0112] Step S4, forming a coil configuration (ie, a recognition algorithm corresponding to a key view angle corresponding to a key image) F = {[Contour_1, Func_1], [Contour_2, Func_2], . . .}.

[0113] The functions of the arbitrary angle matching module are as follows: Assume that there are several key angle configurations in the system, denoted as Config{KA i}, 1<=i<=N. Among them K.A. i is the key view, KA={K,F}, K is the key image corresponding to the key view, and F is the coil configuration corresponding to the key view. The image matching algorithm can also be the image matching algorithm in the prior art, in which the feature extraction can use traditional features such as SURF, or can select deep learning features, etc. In this embodiment, SURF is used as an example for explanation. i}, 1<=i<=all KA in N i The key image K in i And extract point features, such as SURF (or other depth point features), recorded as Feat_i; for any input video frame Frame, extract point features, recorded as Feat0; calculate the similarity distance {dist_i} between Feat0 and all elements of {Feat_i}, select the closest distance, recorded as m, if {dist_i}>Thres (selected threshold), then the key perspective KAm is the matched target key perspective, otherwise it returns unmatched (set m=-1); calculate Feat0 and Feat m The homography matrix H between them is the correction relationship. Correct the coil F in KAm through H to get F'={[H@Contour_1,Func_1],[H@Contour_2,Func_2],···}, and return m,F'.

[0114] Among them, the functions of the key angle selection and overall algorithm flow module are as follows: Key perspective selection, the goal is to adaptively select N better perspectives as key perspectives from the 360-degree perspective of the ball camera according to the stop time of the ball camera. For example, multiple perspectives whose stop time of the ball camera is greater than the preset time can be selected as key perspectives. The upper limit of the number N can be set. For example, at a common intersection, N can be set to 4, and the 4 lanes of the intersection can be observed separately. The specific process is as follows:

[0115] Step S1, when the video analysis system is started, the system coil function configuration is an empty set Config{};

[0116] Step S2: Input video frame Frame in real time, perform “target detection and tracking”, and return to T raj (i, t);

[0117] Step S3, perform "arbitrary angle matching", input video frame Frame, Config{KA i}, return m and F'. If m is -1, continue, indicating that no existing key angle has been matched; otherwise, jump to step S13 for event analysis;

[0118] Step S4, determine whether the current video frame is fixed, and the specific algorithm is not specified. Commonly, for example, the SURF key point features of the upper and lower two frames of the video can be used for matching. If the number of matching points is greater than 100 and the coordinate error of the matching points is less than 3, the video frame can be considered stationary;

[0119] Step S5, if the fixed duration of the video frame is greater than the threshold T (usually set to 10 seconds), continue; otherwise, return to step S2;

[0120] Step S6, record the current time as t0, and select all target trajectories T raj (i, t), and take the trajectory points within the last time T as T raj_T (i, t), where T raj_T (i, t), and eliminate the targets with the distance between the head and tail trajectory points < d (stationary targets, d can be set to 20 pixels);

[0121] Step S7, record the rectangular area in the center of the video frame as S, and the length and width of S are each half of the video frame. Count the number M of T raj_T (i, t) that fall into S. If M > 100, continue; otherwise, return to step S3 and wait;

[0122] Step S8, select the current video frame as the key perspective, and take the average value of the video frame images within the current T as the key frame K;

[0123] Step S9, call the "coil generation" module, input the key image K, and automatically generate the coil configuration F;

[0124] Step S10, record the key angle as KA = {K, F}, put it into the configuration set Config, and get Config{KA i} for 1 <= i <= N;

[0125] Step S11, if the number in Config is greater than N, eliminate one historical key angle KA. Among them, the historical key angle corresponding to the historical video frame with a longer time from the current video frame can be eliminated;

[0126] Step S12, return to step S2;

[0127] Step S13, combine the trajectory T raj (i, t) and F' for event analysis, and report the event result if there is one;

[0128] Step S14, return to step S2.

[0129] The system proposed in this application can truly automatically analyze the key angles that need to be identified (analysis of the trajectory of people and vehicles), and then automatically generate coil configurations to realize event analysis. For any angle, the algorithm can be guaranteed to continue analyzing by matching it to the key angle through automatic correction technology. From a technical perspective, unlike the autonomous driving scenario, the system scenario proposed in this application is a fixed perspective, so that key angles can be extracted, thereby ensuring the stability of the coil configuration; through the correction of any angle, there is no need for a large number of segmentation algorithms, thereby ensuring the efficiency of the analysis.

[0130] The above-mentioned embodiments of the present application can realize the problem of performing event analysis when the ball camera is rotated at any angle. It can not only greatly improve the intelligence capabilities and utilization rate of various cameras, especially ball cameras, but also greatly reduce the cost of manual maintenance, and maintain high efficiency and recognition effect without affecting manual operation and use.

[0131] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including several instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0133] Example 2

[0134] According to an embodiment of the present application, an image recognition method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0135] Figure 4 is a flow chart of an image recognition method according to Embodiment 2 of the present application, such as Figure 4 As shown, the method may include the following steps:

[0136] Step S402, obtaining video frames, wherein the video frames are obtained by shooting the traffic scene with a ball camera at different viewing angles;

[0137] Step S404, tracking the traffic target in the video frame to obtain a first trajectory of the traffic target;

[0138] Step S406, determining a target key perspective that matches the video frame from the multiple key perspectives, wherein different key perspectives correspond to different event analysis algorithms, and different event analysis algorithms are used to analyze traffic events in different traffic areas;

[0139] Step S408, using the event analysis algorithm corresponding to the target key perspective, the traffic event in the traffic scene is analyzed based on the first trajectory to obtain a traffic event analysis result, wherein the traffic event analysis result is used to characterize whether the target traffic event corresponding to the target event analysis algorithm occurs in the traffic scene.

[0140] The above-mentioned traffic targets may be motor vehicles, non-motor vehicles, pedestrians, etc., but are not limited thereto. The above-mentioned target traffic events may include but are not limited to: pedestrians not crossing the road on the zebra crossing, motor vehicles not giving way to pedestrians, and motor vehicles running red lights. The above-mentioned event analysis algorithm may be any one or more algorithms in the prior art that can analyze traffic events to obtain analysis results, and is not limited in this embodiment. The above-mentioned target event analysis algorithm may be an algorithm for identifying the behavior of traffic targets. The above-mentioned traffic event results may be the results such as "motor vehicles not giving way to pedestrians" and "motor vehicles driving in bus lanes" obtained after identifying the behavior of motor vehicles, but are not limited thereto. Among them, the identified traffic targets are different, and the corresponding traffic event analysis results are also different.

[0141] In an optional embodiment, when it is necessary to monitor a traffic scene, first, the traffic scene can be photographed at any angle by a ball camera to obtain a video, and then the ball camera sends the photographed video to a background processing center, which extracts the video photographed by the ball camera to obtain video frames. Secondly, a general target detection and tracking algorithm can be used to track the traffic target and obtain a first trajectory of the traffic target. Then, an image matching algorithm can be used to determine a target key perspective that matches the video frame from multiple key perspectives, and a target event analysis algorithm corresponding to the target key perspective can be used to analyze traffic events in the traffic scene based on the first trajectory to obtain traffic event analysis results, that is, to achieve the purpose of analyzing traffic events in the traffic scene.

[0142] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0143] Example 3

[0144] According to an embodiment of the present application, an image recognition method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0145] Figure 5 is a flow chart of an image recognition method according to Embodiment 3 of the present application, such as Figure 5 As shown, the method may include the following steps:

[0146] Step S502, in response to an input instruction on the operation interface, displaying a video frame on the operation interface, wherein the video frame is a video frame obtained by shooting a real scene at different viewing angles by a ball camera;

[0147] Step S504, in response to the image recognition instruction acting on the operation interface, the target recognition result of the target object in the video frame is displayed on the operation interface, wherein the target recognition result is obtained by identifying the target object based on the first trajectory of the target object using the target recognition algorithm corresponding to the target key perspective, the first trajectory is obtained by tracking the target object in the video frame, the target key perspective is determined from multiple key perspectives to match the video frame, different key perspectives have different corresponding recognition algorithms, different recognition algorithms are used to identify object behaviors in different areas, and the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0148] Figure 6 is a schematic diagram of an optional operation interface according to Example 3 of the present application, such as Figure 6As shown, the operation interface includes: an input instruction input area, an image recognition instruction input area and a display area. When the user wants to determine the behavior of the target object in the video frame, the user can first input an input instruction in the input instruction input area in the operation interface. The operation interface responds to the input instruction and can display the video frame in the operation interface, wherein the video frame is a frame of video obtained by extracting the video from the background processing center after the video is obtained by shooting the real scene by the ball camera according to different viewing angles; secondly, in response to the user inputting the image recognition instruction in the image recognition instruction input area, the operation interface can display the target recognition result of the target object in the video frame in the display area, that is, to achieve the purpose of analyzing the real event in the real scene, wherein the target recognition result can be obtained by using the target recognition algorithm corresponding to the target key viewing angle to recognize the target object based on the first trajectory of the target object, the first trajectory is obtained by tracking the target object in the video frame, the target key viewing angle is determined from multiple key viewing angles to match the video frame, different key viewing angles correspond to different recognition algorithms, different recognition algorithms are used to recognize the object behavior in different areas, and the target recognition result is used to characterize whether the target object performs the target behavior corresponding to the target recognition algorithm.

[0149] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0150] Example 4

[0151] According to an embodiment of the present application, there is also provided an image recognition method that can be applied to virtual reality scenarios such as virtual reality VR devices and augmented reality AR devices. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0152] Figure 7 4 is a flowchart of an image recognition method according to Embodiment 4 of the present application. Figure 7 As shown, the method may include the following steps:

[0153] Step S702, displaying video frames on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the video frames are obtained by shooting a real scene with a ball camera at different viewing angles;

[0154] Step S704, tracking the target object in the video frame to obtain a first trajectory of the target object;

[0155] Step S706, determining a target key perspective that matches the video frame from the multiple key perspectives, wherein different key perspectives correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas;

[0156] Step S708, using a target recognition algorithm corresponding to the target key perspective to recognize the target object based on the first trajectory, and obtaining a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object performs a target behavior corresponding to the target recognition algorithm;

[0157] Step S7010: drive the VR device or AR device to render and display the target recognition result.

[0158] In an optional embodiment, when a user needs to determine the behavior of a target object in a video frame, first, the video frame can be displayed on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the video frame is a frame of video obtained by extracting a video obtained by shooting a real scene from different perspectives with a ball camera; secondly, the target object in the video frame can be tracked by a general target detection and tracking algorithm to obtain a first trajectory of the target object; then, the target key perspective that matches the video frame can be determined from multiple key perspectives by an image matching algorithm, and the target object can be identified based on the first trajectory by a target recognition algorithm corresponding to the target key perspective to obtain a target recognition result of the target object, and finally, the VR device or AR device can be driven to render and display the target recognition result, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm. Among them, the recognition algorithms corresponding to different key perspectives are different, and different recognition algorithms are used to identify the behavior of objects in different areas.

[0159] Optionally, in this embodiment, the above-mentioned image recognition method can be applied to a hardware environment composed of a server and a virtual reality device. The target recognition result is displayed on the presentation screen of the virtual reality VR device or the augmented reality AR device. The server can be a server corresponding to the media file operator. The above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network. The above-mentioned virtual reality device is not limited to: a virtual reality helmet, a virtual reality glasses, a virtual reality all-in-one machine, etc.

[0160] Optionally, the virtual reality device includes: a memory, a processor, and a transmission device. The memory is used to store an application, which can be used to execute: displaying a video frame on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the video frame is a video frame obtained by shooting a real scene at different viewing angles by a ball camera; tracking a target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas; using a target recognition algorithm corresponding to the target key viewing angle, based on the first trajectory, recognize the target object to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; driving the VR device or AR device to render and display the target recognition result.

[0161] It should be noted that the image recognition method applied in the VR device or AR device of this embodiment may include: Figure 7 The method of the illustrated embodiment is used to achieve the purpose of driving a VR device or an AR device to display target recognition results.

[0162] Optionally, the processor of this embodiment can call the application stored in the memory to execute the above steps through a transmission device. The transmission device can receive media files sent by the server through the network, and can also be used for data transmission between the processor and the memory.

[0163] Optionally, in a virtual reality device, a head-mounted display with eye tracking is provided, wherein the screen in the HMD is used to display the displayed video images, the eye tracking module in the HMD is used to obtain the real-time movement trajectory of the user's eyes, the tracking system is used to track the user's position information and motion information in the real three-dimensional space, and the computing processing unit is used to obtain the user's real-time position and motion information from the tracking system, and calculate the three-dimensional coordinates of the user's head in the virtual three-dimensional space, as well as the user's field of view direction in the virtual three-dimensional space, etc.

[0164] In an embodiment of the present application, a virtual reality device may be connected to a terminal, and the terminal and the server are connected through a network. The virtual reality device is not limited to: a virtual reality helmet, virtual reality glasses, a virtual reality all-in-one machine, etc. The terminal is not limited to a PC, a mobile phone, a tablet computer, etc. The server may be a server corresponding to a media file operator, and the network includes but is not limited to: a wide area network, a metropolitan area network or a local area network.

[0165] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0166] Example 5

[0167] According to an embodiment of the present application, an image recognition method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0168] Figure 8 is a flow chart of an image recognition method according to Embodiment 5 of the present application, such as Figure 8 As shown, the method may include the following steps:

[0169] Step S802, obtaining a video frame by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter is a video frame, and the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles;

[0170] Step S804, tracking the target object in the video frame to obtain a first trajectory of the target object;

[0171] Step S806, determining a target key perspective that matches the video frame from the multiple key perspectives, wherein different key perspectives correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas;

[0172] Step S808, using a target recognition algorithm corresponding to the target key perspective to recognize the target object based on the first trajectory, and obtaining a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object performs a target behavior corresponding to the target recognition algorithm;

[0173] Step S8010: output the target recognition result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target recognition result.

[0174] The first interface mentioned above may be an interface for acquiring video frames from a server, and the second interface mentioned above may be an interface for sending target recognition results to the server.

[0175] In an optional embodiment, when a user wants to determine the behavior of a target object in a video, the user can first obtain a video frame from a server by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is a video frame, and the video frame is a video frame obtained by shooting a real scene with a ball camera at different viewing angles; secondly, the target object in the video frame can be tracked by a general target detection and tracking algorithm to obtain a first trajectory of the target object; then, a target key viewing angle matching the video frame can be determined from multiple key viewing angles by an image matching algorithm, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify the behavior of objects in different areas; then, the target recognition algorithm corresponding to the target key viewing angle can be used to identify the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; finally, the target recognition result can be output by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target recognition result.

[0176] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0177] Example 6

[0178] According to an embodiment of the present application, an image recognition device for implementing the above-mentioned image recognition method is also provided. Fig. 9 is a schematic diagram of an image recognition device according to Embodiment 6 of the present application, such as Fig. 9 As shown, the device includes: an acquisition module 90, a tracking module 92, a determination module 94 and an identification module 96.

[0179] Among them, the acquisition module is used to acquire video frames, wherein the video frames are obtained by shooting real scenes with a ball camera according to different viewing angles; the tracking module is used to track the target object in the video frame to obtain a first trajectory of the target object; the determination module is used to determine the target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; the recognition module is used to use the target recognition algorithm corresponding to the target key viewing angle to identify the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0180] It should be noted that the acquisition module 90, tracking module 92, determination module 94 and identification module 96 correspond to steps S302 to S308 in Example 1, and the four modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the modules may also be part of a device and run in the AR / VR device provided in Example 1.

[0181] In the above embodiment of the present application, the determination module includes: an extraction unit, an acquisition unit and a first determination unit.

[0182] Among them, the extraction unit is used to perform feature extraction on key images corresponding to multiple key perspectives to obtain multiple first pixel features, and to perform feature extraction on video frames to obtain second pixel features, and the multiple first pixel features correspond one-to-one to the multiple key perspectives; the acquisition unit is used to obtain the distance between the multiple first pixel features and the second pixel features to obtain multiple distances; the first determination unit is used to determine that the key perspective corresponding to the minimum distance among the multiple distances is the target key perspective.

[0183] In the above embodiments of the present application, the determining unit includes: a first determining subunit and a second determining subunit.

[0184] Among them, the first determination subunit is used to determine whether the minimum distance is greater than a preset threshold; the second determination subunit is used to determine that the key perspective corresponding to the minimum distance is the target key perspective when the minimum distance is less than or equal to the preset threshold.

[0185] In the above embodiment of the present application, the determining unit further includes: a third determining subunit, a fourth determining subunit, a constructing subunit and a storing subunit.

[0186] Among them, the third determination subunit is used to determine whether the target perspective corresponding to the video frame is a key perspective based on the first trajectory; the fourth determination subunit is used to determine the key image corresponding to the target perspective based on multiple video frames within a preset time period when the target perspective is the key perspective, wherein the preset time period is a time period before the current moment and the length is a preset length; the construction subunit is used to construct a recognition algorithm corresponding to the target perspective based on the key image corresponding to the target perspective; the storage subunit is used to store the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective in a corresponding manner.

[0187] In the above-mentioned embodiment of the present application, the third determination subunit is also used to: determine whether the video frame is a fixed picture; in the case that the video frame is a fixed picture, determine whether the fixed duration corresponding to the fixed picture is greater than the preset duration, wherein the fixed duration is used to characterize the duration of the time structure corresponding to the fixed picture; in the case that the fixed duration is greater than the preset duration, determine the number of targets located in the preset area of ​​the video frame among the trajectory points included in the first trajectory; in the case that the number of targets is greater than the preset number, determine that the target perspective is a critical perspective; in the case that the video frame is not a fixed picture, or the fixed duration is less than or equal to the preset duration, or the number of targets is less than or equal to the preset number, determine that the target perspective is not a critical perspective.

[0188] In the above embodiment of the present application, the third determination subunit is also used to: obtain a second trajectory located within a preset time period in the first trajectory; determine the moving distance of the target object based on the second trajectory; eliminate target objects whose moving distance is less than the preset distance from the second trajectory to obtain a third trajectory; determine the number of targets located in the preset area among the trajectory points contained in the third trajectory.

[0189] In the above embodiment of the present application, the fourth determining subunit is further used to: obtain an average value of multiple video frames and a key image corresponding to the target viewing angle.

[0190] In the above embodiments of the present application, the construction subunit is also used to: perform image segmentation on the key image corresponding to the target perspective to obtain at least one semantic area; perform contour extraction on at least one semantic area to obtain a region contour corresponding to at least one semantic area; determine a recognition algorithm corresponding to at least one semantic area from multiple preset recognition algorithms; and summarize the recognition algorithms corresponding to at least one semantic area to obtain a recognition algorithm corresponding to the target perspective.

[0191] In the above embodiments of the present application, the storage subunit is also used to: determine whether the number of multiple key perspectives is greater than or equal to a preset number; when the number of multiple key perspectives is greater than or equal to the preset number, eliminate the historical key perspectives, the key images corresponding to the historical key perspectives, and the recognition algorithms corresponding to the historical key perspectives among the multiple key perspectives; and store the target perspective, the key images corresponding to the target perspective, and the recognition algorithms corresponding to the target perspective accordingly.

[0192] In the above embodiments of the present application, the identification module includes: a second determination unit, a correction unit and an identification unit.

[0193] Among them, the second determination unit is used to determine the homography matrix between the target key image corresponding to the target key perspective and the video frame; the correction unit is used to correct the target area corresponding to the target recognition algorithm based on the homography matrix to obtain a corrected recognition algorithm; the recognition unit is used to use the corrected recognition algorithm to identify the target object based on the first trajectory to obtain a target recognition result.

[0194] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0195] Example 7

[0196] According to an embodiment of the present application, an image recognition device for implementing the above-mentioned image recognition method is also provided. Fig.10 is a schematic diagram of an image recognition device according to Embodiment 7 of the present application, such as Fig.10 As shown, the device includes: an acquisition module 1002 , a tracking module 1004 , a determination module 1006 and an analysis module 1008 .

[0197] The acquisition module is used to acquire video frames, wherein the video frames are obtained by shooting traffic scenes with a ball camera according to different viewing angles; the tracking module is used to track traffic targets in the video frames to obtain a first trajectory of the traffic targets; the determination module is used to determine a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different event analysis algorithms, and different event analysis algorithms are used to analyze traffic events in different traffic areas; the analysis module is used to use the event analysis algorithm corresponding to the target key viewing angle to analyze traffic events in the traffic scene based on the first trajectory to obtain a traffic event analysis result, wherein the traffic event analysis result is used to characterize whether a target traffic event corresponding to the target event analysis algorithm occurs in the traffic scene.

[0198] It should be noted that the acquisition module 1002, tracking module 1004, determination module 1006 and analysis module 1008 correspond to steps S402 to S408 in Example 2, and the four modules and corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in Example 2. It should be noted that the modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the modules may also be part of a device and run in the AR / VR device provided in Example 1.

[0199] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0200] Example 8

[0201] According to an embodiment of the present application, an image recognition device for implementing the above-mentioned image recognition method is also provided. Fig.11 is a schematic diagram of an image recognition device according to Embodiment 8 of the present application, such as Fig.11 As shown, the device includes: a first display module 1102 and a second display module 1104 .

[0202] Among them, the first display module is used to respond to input instructions acting on the operation interface, and display video frames on the operation interface, wherein the video frames are video frames obtained by shooting real scenes with a ball camera according to different viewing angles; the second display module is used to respond to image recognition instructions acting on the operation interface, and display target recognition results of target objects in video frames on the operation interface, wherein the target recognition results are obtained by using a target recognition algorithm corresponding to a target key viewing angle to identify the target object based on a first trajectory of the target object, the first trajectory is obtained by tracking the target object in the video frame, the target key viewing angle is a key viewing angle that matches the video frame determined from multiple key viewing angles, different key viewing angles have different corresponding recognition algorithms, different recognition algorithms are used to identify object behaviors in different areas, and the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0203] It should be noted that the first display module 1102 and the second display module 1104 correspond to steps S502 to S504 in Example 3, and the two modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 3. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and run in the AR / VR device provided in Example 1.

[0204] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0205] Example 9

[0206] According to an embodiment of the present application, an image recognition device for implementing the above-mentioned image recognition method is also provided. Fig.12 is a schematic diagram of an image recognition device according to Embodiment 9 of the present application, such as Fig.12 As shown, the device includes: a first display module 1202 , a tracking module 1204 , a determination module 1206 , an identification module 1208 and a second display module 12010 .

[0207] Among them, the first display module is used to display video frames on the presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the video frames are obtained by shooting real scenes with a ball camera according to different viewing angles; the tracking module is used to track the target object in the video frame to obtain a first trajectory of the target object; the determination module is used to determine the target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; the recognition module is used to use the target recognition algorithm corresponding to the target key viewing angle to identify the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; the second display module is used to drive the VR device or AR device to render and display the target recognition result.

[0208] It should be noted that the first display module 1202, tracking module 1204, determination module 1206, identification module 1208 and second display module 12010 correspond to steps S702 to S7010 in Example 4, and the five modules and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Example 4. It should be noted that the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above modules may also be part of a device and run in the AR / VR device provided in Example 1.

[0209] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0210] Example 10

[0211] According to an embodiment of the present application, an image recognition device for implementing the above-mentioned image recognition method is also provided. Fig.13 is a schematic diagram of an image recognition device according to Embodiment 10 of the present application, such as Fig.13 As shown, the device includes: an acquisition module 1302 , a tracking module 1304 , a determination module 1306 , an identification module 1308 and an output module 13010 .

[0212] Among them, the acquisition module is used to obtain video frames by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is a video frame, and the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; the tracking module is used to track the target object in the video frame to obtain the first trajectory of the target object; the determination module is used to determine the target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; the recognition module is used to use the target recognition algorithm corresponding to the target key viewing angle to identify the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; the output module is used to output the target recognition result by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the target recognition result.

[0213] It should be noted that the acquisition module 1302, tracking module 1304, determination module 1306, identification module 1308 and output module 13010 correspond to steps S802 to S8010 in Example 5, and the five modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 5. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above-mentioned modules may also be part of the device and run in the AR / VR device provided in Example 1.

[0214] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0215] Embodiment 11

[0216] The embodiment of the present application may provide an AR / VR device, which may be any AR / VR device in an AR / VR device group. Optionally, in this embodiment, the AR / VR device may also be replaced by a terminal device such as a mobile terminal.

[0217] Optionally, in this embodiment, the AR / VR device may be located in at least one network device among a plurality of network devices of a computer network.

[0218] In this embodiment, the above-mentioned AR / VR device can execute the program code of the following steps in the image recognition method: displaying a video frame on a presentation screen of a virtual reality VR device or an augmented reality AR device, wherein the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; tracking the target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; using the target recognition algorithm corresponding to the target key viewing angle, identifying the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm; driving the VR device or AR device to render and display the target recognition result.

[0219] Optionally, Fig.14 is a structural block diagram of a computer terminal according to an embodiment of the present application. Fig.14 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1402, a memory 1404, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0220] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the image recognition method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image recognition method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0221] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a video frame, wherein the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; track the target object in the video frame to obtain a first trajectory of the target object; determine a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; use the target recognition algorithm corresponding to the target key viewing angle to identify the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0222] Optionally, the processor may also execute the program code of the following steps: performing feature extraction on key images corresponding to multiple key perspectives to obtain multiple first pixel features, and performing feature extraction on video frames to obtain second pixel features, wherein the multiple first pixel features correspond one-to-one to the multiple key perspectives; obtaining the distances between the multiple first pixel features and the second pixel features to obtain multiple distances; and determining that the key perspective corresponding to the minimum distance among the multiple distances is the target key perspective.

[0223] Optionally, the processor may also execute program code of the following steps: determining whether the minimum distance is greater than a preset threshold; and when the minimum distance is less than or equal to the preset threshold, determining the key viewing angle corresponding to the minimum distance as the target key viewing angle.

[0224] Optionally, the processor may also execute the program code of the following steps: based on the first trajectory, determining whether the target perspective corresponding to the video frame is a key perspective; if the target perspective is a key perspective, determining a key image corresponding to the target perspective based on multiple video frames within a preset time period, wherein the preset time period is a time period before the current moment and has a preset length; based on the key image corresponding to the target perspective, constructing a recognition algorithm corresponding to the target perspective; and storing the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective in correspondence.

[0225] Optionally, the processor may also execute the following program code: determining whether the video frame is a fixed picture; in the case that the video frame is a fixed picture, determining whether the fixed duration corresponding to the fixed picture is greater than a preset duration, wherein the fixed duration is used to characterize the duration of the time structure corresponding to the fixed picture; in the case that the fixed duration is greater than the preset duration, determining the number of targets located in a preset area of ​​the video frame among the trajectory points included in the first trajectory; in the case that the number of targets is greater than the preset number, determining that the target perspective is a critical perspective; in the case that the video frame is not a fixed picture, or the fixed duration is less than or equal to the preset duration, or the number of targets is less than or equal to the preset number, determining that the target perspective is not a critical perspective.

[0226] Optionally, the processor may also execute program code of the following steps: obtaining a second trajectory in the first trajectory that is within a preset time period; determining a moving distance of the target object based on the second trajectory; eliminating target objects whose moving distance is less than a preset distance from the second trajectory to obtain a third trajectory; and determining the number of targets located in a preset area among the trajectory points included in the third trajectory.

[0227] Optionally, the processor may also execute program code of the following steps: obtaining an average value of multiple video frames and a key image corresponding to a target viewing angle.

[0228] Optionally, the processor may also execute program codes of the following steps: performing image segmentation on a key image corresponding to a target perspective to obtain at least one semantic area; performing contour extraction on at least one semantic area to obtain a region contour corresponding to at least one semantic area; determining a recognition algorithm corresponding to at least one semantic area from a plurality of preset recognition algorithms; and summarizing the recognition algorithms corresponding to at least one semantic area to obtain a recognition algorithm corresponding to the target perspective.

[0229] Optionally, the processor may also execute the program code of the following steps: determining whether the number of multiple key perspectives is greater than or equal to a preset number; when the number of multiple key perspectives is greater than or equal to a preset number, eliminating historical key perspectives, key images corresponding to the historical key perspectives, and recognition algorithms corresponding to the historical key perspectives from the multiple key perspectives; and storing the target perspective, key images corresponding to the target perspective, and recognition algorithms corresponding to the target perspective in correspondence.

[0230] Optionally, the processor may also execute the program code of the following steps: determining the homography matrix between the target key image and the video frame corresponding to the target key perspective; correcting the target area corresponding to the target recognition algorithm based on the homography matrix to obtain a corrected recognition algorithm; and using the corrected recognition algorithm to identify the target object based on the first trajectory to obtain a target recognition result.

[0231] By adopting the embodiment of the present application, a method is provided for obtaining a video frame; tracking a target object in the video frame to obtain a first trajectory of the target object; determining a target key perspective that matches the video frame from multiple key perspectives; using a target recognition algorithm corresponding to the target key perspective to recognize the target object based on the first trajectory to obtain a target recognition result of the target object, thereby achieving the purpose of identifying the behavior of the target object through a ball camera. It is easy to notice that since the ball camera can obtain video frames according to different perspectives, by determining the target key perspective that matches the video frame, and using a target recognition algorithm corresponding to the target key perspective to recognize the target object based on the first trajectory, it is possible to automatically analyze the key angles that need to be identified based on any angle, and quickly and accurately obtain the target recognition result of the target object, thereby achieving the technical effect of improving the intelligence and efficiency of the ball camera's video analysis, and thus solving the technical problem of low intelligence and low efficiency of the ball camera's video analysis caused by manual processing of the video obtained by the ball camera in the related art.

[0232] Those skilled in the art will appreciate that the structure shown in the figure is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Fig.14 The structure of the electronic device is not limited. For example, the computer terminal A may also include Fig.14 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.14 Different configurations shown.

[0233] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0234] Example 12

[0235] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the image recognition method provided in the above embodiment 1.

[0236] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be located in any computer terminal in the AR / VR device terminal group in the AR / VR device network, or in any mobile terminal in the mobile terminal group.

[0237] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for executing the following steps: acquiring a video frame, wherein the video frame is obtained by shooting a real scene with a ball camera at different viewing angles; tracking a target object in the video frame to obtain a first trajectory of the target object; determining a target key viewing angle that matches the video frame from multiple key viewing angles, wherein different key viewing angles correspond to different recognition algorithms, and different recognition algorithms are used to identify object behaviors in different areas; using a target recognition algorithm corresponding to the target key viewing angle, identifying the target object based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

[0238] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: performing feature extraction on key images corresponding to multiple key perspectives to obtain multiple first pixel features, and performing feature extraction on video frames to obtain second pixel features, wherein the multiple first pixel features correspond one-to-one to the multiple key perspectives; obtaining the distance between the multiple first pixel features and the second pixel features to obtain multiple distances; and determining that the key perspective corresponding to the minimum distance among the multiple distances is the target key perspective.

[0239] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for executing the following steps: determining whether the minimum distance is greater than a preset threshold; when the minimum distance is less than or equal to the preset threshold, determining the key perspective corresponding to the minimum distance as the target key perspective.

[0240] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: based on the first trajectory, determining whether the target perspective corresponding to the video frame is a key perspective; if the target perspective is a key perspective, determining a key image corresponding to the target perspective based on multiple video frames within a preset time period, wherein the preset time period is a time period before the current moment and has a preset length; based on the key image corresponding to the target perspective, constructing a recognition algorithm corresponding to the target perspective; and storing the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective in correspondence.

[0241] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: determining whether the video frame is a fixed picture; in the case that the video frame is a fixed picture, determining whether the fixed duration corresponding to the fixed picture is greater than a preset duration, wherein the fixed duration is used to characterize the duration of the time structure corresponding to the fixed picture; in the case that the fixed duration is greater than the preset duration, determining the number of targets located in a preset area of ​​the video frame among the trajectory points included in the first trajectory; in the case that the number of targets is greater than the preset number, determining that the target perspective is a critical perspective; in the case that the video frame is not a fixed picture, or the fixed duration is less than or equal to the preset duration, or the number of targets is less than or equal to the preset number, determining that the target perspective is not a critical perspective.

[0242] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program codes for executing the following steps: obtaining a second trajectory in the first trajectory that is within a preset time period; determining a moving distance of the target object based on the second trajectory; eliminating target objects whose moving distance is less than a preset distance from the second trajectory to obtain a third trajectory; and determining the number of targets located in a preset area among the trajectory points included in the third trajectory.

[0243] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program codes for executing the following steps: obtaining an average value of multiple video frames and a key image corresponding to a target viewing angle.

[0244] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: performing image segmentation on the key image corresponding to the target perspective to obtain at least one semantic area; performing contour extraction on at least one semantic area to obtain a region contour corresponding to at least one semantic area; determining a recognition algorithm corresponding to at least one semantic area from multiple preset recognition algorithms; and summarizing the recognition algorithms corresponding to at least one semantic area to obtain a recognition algorithm corresponding to the target perspective.

[0245] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: determining whether the number of multiple key perspectives is greater than or equal to a preset number; when the number of multiple key perspectives is greater than or equal to a preset number, eliminating historical key perspectives, key images corresponding to the historical key perspectives, and recognition algorithms corresponding to the historical key perspectives from the multiple key perspectives; and storing the target perspective, the key image corresponding to the target perspective, and the recognition algorithm corresponding to the target perspective accordingly.

[0246] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: determining the homography matrix between the target key image and the video frame corresponding to the target key perspective; correcting the target area corresponding to the target recognition algorithm based on the homography matrix to obtain a corrected recognition algorithm; and using the corrected recognition algorithm to identify the target object based on the first trajectory to obtain a target recognition result.

[0247] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0248] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0249] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0250] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0251] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0252] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0253] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image recognition method, characterized in that: include: Acquire video frames, wherein the video frames are obtained by shooting a real scene with a ball camera at different viewing angles; Tracking the target object in the video frame to obtain a first trajectory of the target object; Determining a target key perspective that matches the video frame from a plurality of key perspectives, wherein different key perspectives correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas; The target object is identified based on the first trajectory using a target recognition algorithm corresponding to the target key perspective to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

2. The method according to claim 1, characterized in that: Determining a target key perspective that matches the video frame from a plurality of key perspectives includes: Performing feature extraction on key images corresponding to the multiple key perspectives to obtain multiple first pixel features, and performing feature extraction on the video frame to obtain second pixel features, wherein the multiple first pixel features correspond one-to-one to the multiple key perspectives; Acquire distances between the plurality of first pixel features and the second pixel features to obtain a plurality of distances; A key viewing angle corresponding to a minimum distance among the multiple distances is determined as the target key viewing angle.

3. The method according to claim 2, characterized in that Determining a key viewing angle corresponding to a minimum distance among the multiple distances as the target key viewing angle includes: Determining whether the minimum distance is greater than a preset threshold; When the minimum distance is less than or equal to the preset threshold, the key viewing angle corresponding to the minimum distance is determined as the target key viewing angle.

4. The method according to claim 3, characterized in that When the minimum distance is greater than the preset threshold, the method further includes: Based on the first trajectory, determining whether the target perspective corresponding to the video frame is the key perspective; In the case where the target viewing angle is the key viewing angle, determining a key image corresponding to the target viewing angle based on a plurality of video frames within a preset time period, wherein the preset time period is a time period before the current moment and has a preset length; Based on the key image corresponding to the target perspective, construct a recognition algorithm corresponding to the target perspective; The target viewing angle, the key image corresponding to the target viewing angle, and the recognition algorithm corresponding to the target viewing angle are stored correspondingly.

5. The method according to claim 4, characterized in that Determining, based on the first trajectory, whether a target perspective corresponding to the video frame is the key perspective includes: Determining whether the video frame is a fixed picture; In the case where the video frame is the fixed picture, determining whether a fixed duration corresponding to the fixed picture is greater than a preset duration, wherein the fixed duration is used to represent the duration of the time structure corresponding to the fixed picture; When the fixed time length is greater than the preset time length, determining the number of targets located in a preset area of ​​the video frame among the track points included in the first track; When the target number is greater than a preset number, determining that the target viewing angle is the key viewing angle; When the video frame is not a fixed picture, or the fixed duration is less than or equal to the preset duration, or the target number is less than or equal to the preset number, it is determined that the target viewing angle is not the key viewing angle.

6. The method according to claim 5, characterized in that Determining the number of targets located in a preset area of ​​the video frame among the trajectory points included in the first trajectory includes: Acquire a second trajectory in the first trajectory that is within the preset time period; Based on the second trajectory, determining a moving distance of the target object; Eliminate the target objects whose moving distance is less than a preset distance from the second trajectory to obtain a third trajectory; Determine the number of targets located within the preset area among the trajectory points included in the third trajectory.

7. The method according to claim 4, characterized in that Based on the key image corresponding to the target perspective, a recognition algorithm corresponding to the target perspective is constructed, including: Performing image segmentation on the key image corresponding to the target perspective to obtain at least one semantic region; Performing contour extraction on the at least one semantic region to obtain a region contour corresponding to the at least one semantic region; Determining a recognition algorithm corresponding to the at least one semantic area from a plurality of preset recognition algorithms; The recognition algorithm corresponding to the at least one semantic area is summarized to obtain a recognition algorithm corresponding to the target perspective.

8. The method according to claim 1, characterized in that Using the target recognition algorithm corresponding to the target key perspective, the target object is recognized based on the first trajectory to obtain a target recognition result of the target object, including: Determine a homography matrix between a target key image corresponding to the target key perspective and the video frame; Correcting the target area corresponding to the target recognition algorithm based on the homography matrix to obtain a corrected recognition algorithm; The target object is identified based on the first trajectory using the modified identification algorithm to obtain the target identification result.

9. An image recognition method, characterized in that: include: Acquire video frames, wherein the video frames are obtained by shooting the traffic scene with a ball camera at different viewing angles; Tracking the traffic target in the video frame to obtain a first trajectory of the traffic target; Determining a target key perspective that matches the video frame from a plurality of key perspectives, wherein different key perspectives correspond to different event analysis algorithms, and different event analysis algorithms are used to analyze traffic events in different traffic areas; Using the event analysis algorithm corresponding to the target key perspective, the traffic events in the traffic scene are analyzed based on the first trajectory to obtain a traffic event analysis result, wherein the traffic event analysis result is used to characterize whether the target traffic event corresponding to the target event analysis algorithm occurs in the traffic scene.

10. An image recognition method, characterized in that: include: In response to an input instruction acting on the operation interface, a video frame is displayed on the operation interface, wherein the video frame is a video frame obtained by shooting a real scene at different viewing angles by a ball camera; In response to an image recognition instruction applied to the operation interface, a target recognition result of the target object in the video frame is displayed on the operation interface, wherein the target recognition result is obtained by identifying the target object based on a first trajectory of the target object using a target recognition algorithm corresponding to a target key perspective, the first trajectory being obtained by tracking the target object in the video frame, the target key perspective being a key perspective that matches the video frame determined from multiple key perspectives, different key perspectives corresponding to different recognition algorithms, different recognition algorithms being used to identify object behaviors in different areas, and the target recognition result being used to characterize whether the target object executes the target behavior corresponding to the target recognition algorithm.

11. An image recognition method, characterized in that: include: Displaying video frames on a presentation screen of a virtual reality (VR) device or an augmented reality (AR) device, wherein the video frames are video frames obtained by shooting a real scene at different viewing angles using a ball camera; Tracking the target object in the video frame to obtain a first trajectory of the target object; Determining a target key perspective that matches the video frame from a plurality of key perspectives, wherein different key perspectives correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas; Using a target recognition algorithm corresponding to the target key perspective, the target object is recognized based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object performs a target behavior corresponding to the target recognition algorithm; The VR device or the AR device is driven to render and display the target recognition result.

12. An image recognition method, characterized in that: include: Acquire a video frame by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter is the video frame, and the video frame is a video frame obtained by shooting a real scene with a ball camera according to different viewing angles; Tracking the target object in the video frame to obtain a first trajectory of the target object; Determining a target key perspective that matches the video frame from a plurality of key perspectives, wherein different key perspectives correspond to different recognition algorithms, and different recognition algorithms are used to recognize object behaviors in different areas; Using a target recognition algorithm corresponding to the target key perspective, the target object is recognized based on the first trajectory to obtain a target recognition result of the target object, wherein the target recognition result is used to characterize whether the target object performs a target behavior corresponding to the target recognition algorithm; The target recognition result is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the target recognition result.

13. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 12 when running.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

Images