Method for detecting interactions between several elements of interest

Abstract

The invention relates to a method for detecting interactions between several elements of interest (P1, P2, P3) using a computer, based on a plurality of images comprising the elements of interest. The images of the plurality of images are acquired at different times. The method comprises, for each image, a characterization step in which the computer characterizes, for each element of interest, an orientation of the element of interest, and a definition step in which the computer defines, for each element of interest, a membership space (20) based on the characterized orientation of the element of interest. The method also comprises an interaction detection step in which the computer detects interactions between elements of interest (P1, P2, P3) based on the relative location of the membership spaces (20) of the elements of interest with respect to each other in time. (See Figure 2 for the abstract.)

Description

Title of the invention: Method for detecting interactions between several elements of interest

[0001] The invention relates to the field of detecting interactions between elements of interest, in particular people in vehicles or on foot, or animals, for example with the aim of verifying compliance with a distance between said elements of interest or of detecting exchanges, communications, interactions between said elements of interest.

[0002] It is known from the state of the art to detect objects or people in images, to follow them over time, to predict their trajectory and / or their collision, to estimate distances separating them. The objects or people are represented by boxes or skeletons to facilitate their processing by a computer. For example, the distances between the objects are determined by means of a measurement of the distances between the corresponding boxes. The boxes can be represented superimposed in the images, in two or three dimensions.

[0003] Sometimes, circles are associated with the elements of interest, a circle centered on an element of interest marking a territorial boundary around said element of interest. Such a territorial boundary makes it possible, for example, to represent a social distance to be respected. The circles are represented in the images in three dimensions or approximated as ellipses in two dimensions. These circles do not make it possible to reliably detect interactions between several elements of interest because the orientation of the elements of interest is not taken into account.

[0004] The invention aims to solve the above-mentioned problems of the state of the art by proposing a method.

[0005] The invention relates to a method for detecting interactions between several elements of interest, by a computer, from a plurality of images comprising the elements of interest, the images of the plurality of images being acquired at different times in time, the method comprising for each image the following steps: - a characterization step, in which the computer characterizes for each element of interest, an orientation of the element of interest, - a definition step, in which the computer defines for each element of interest, a membership space as a function of the orientation of the characterized element of interest, and comprising an interaction detection step, in which the computer detects interactions between elements of interest as a function of the location relative membership spaces of the elements of interest in relation to each other over time.

[0006] According to one aspect of the invention, at least one element of interest is a person and the method for detecting interactions comprises, for each element of interest being a person, a selection step in which the calculator selects characteristic points defining the element of interest, the points comprising a pair of points relating to shoulders, and at least one other pair of points from among a pair of points relating to wrists and a pair of points relating to elbows, the orientation being characterized according to a calculation depending on the pair of points relating to the shoulders and the at least one other pair of points.

[0007] According to one aspect of the invention, the calculator selects the three pairs of points comprising the pair of points relating to shoulders, the pair of points relating to wrists and the pair of points relating to elbows, the orientation being characterized according to a calculation based on the three pairs of points and weighted by weights, a first weight being attributed to the pair of points relating to the shoulders, a second weight being attributed to the pair of points relating to the elbows and a third weight being attributed to the pair of points relating to the wrists, the first weight being higher than the second weight, the second weight being higher than the third weight.

[0008] According to one aspect of the invention, the orientation comprises an orientation vector, the membership space of an element of interest being defined as a function of an anchor point referenced relative to the element of interest and a reference vector passing through the anchor point, parallel to the orientation vector and in the same direction, the membership space comprising limits passing through the anchor point and forming a salient angle of which the reference vector is a bisector and the anchor point the vertex.

[0009] According to one aspect of the invention, the interaction detection step comprises determining intersections between the membership spaces of the elements of interest, an intersection between two membership spaces corresponding to a connection between the two elements of interest associated with said two membership spaces, an interaction being detected as a function of several connections over time between said two elements of interest.

[0010] According to one aspect of the invention, each membership space is divided into a plurality of zones, a connection class between the two elements of interest being assigned based on a location of intersection of the zones of the two membership spaces.

[0011] According to one aspect of the invention, an interaction is detected if a predetermined number of connections between said two elements of interest is detected over a predefined number of successive images over time.

[0012] According to one aspect of the invention, the method further comprises a step of establishing a list in which the calculator establishes a list of elements of interest having entered into interaction with a predetermined element of interest as a function of detected interactions.

[0013] According to one aspect of the invention, the method further comprises for each image, a display step in which the calculator displays superimposed in the image, for each element of interest, a delimitation of the membership space and a representation of the detected interactions if interactions have been detected.

[0014] According to one aspect of the invention, the display step further comprises for each element of interest the display of a box surrounding the element of interest.

[0015] According to one aspect of the invention, the display step further comprising for each element of interest the display of an orientation indicator depending on the characterized orientation.

[0016] The invention also relates to a computer program product comprising the program instructions implementing the steps of the matching method according to any one of claims 1 to 9, when the program instructions are executed by a computer.

[0017] Other advantages and characteristics of the invention will appear on reading the description and the drawings.

[0018] [Fig-1] represents a modeled person and associated a space of belonging.

[0019] [Fig.2] represents three persons as illustrated in [Fig.l], an interaction being detected between two of the three people.

[0020] [Fig.3] illustrates the steps of the method for detecting interactions between several elements of interest, according to the invention.

[0021] [Fig.4] illustrates a system capable of implementing the method of the invention.

[0022] [Fig.l] represents an image comprising an element of interest P which is a person.

[0023] For example, as shown in [Fig. 1], the element of interest P is modeled by a skeleton S and defined by characteristic points 50, 51, 52a, 52b, 53a, 53b, 54a, 54b.

[0024] The characteristic points include: - a foot point 50, which is located on the ground in the middle of a left foot point of skeleton S and a right foot point of skeleton S, - a head point 51 of the skeleton S, - a left shoulder point 52a of the skeleton S and a right shoulder point 52b of the skeleton S forming a pair of points relative to the shoulders (52a, 52b), - a left elbow point 53a of the skeleton S and a right elbow point 53b of the skeleton S forming a pair of points relative to the elbows (53a, 53b), - a left wrist point 54a of skeleton S and a right wrist point 54b of skeleton S forming a pair of points relating to the shoulders (54a, 54b).

[0025] The element of interest P is further modeled by an enveloping box 30 in the shape of a right circular cylinder relative to the ground of the image.

[0026] For example, the enveloping box 30 is three-dimensional. In the image, the perspective effect may distort the enveloping box 30, giving the impression of an oblique elliptical cylinder shape.

[0027] For example, the wrap-around box 30 is two-dimensional.

[0028] The enveloping box 30 comprises a lower base BI located in the ground plane of the image and comprising the foot point 50, and an upper base BS comprising the head point 51.

[0029] The lower base BI and the upper base BS are discs.

[0030] Advantageously, the radius of the lower base BI and the upper base BS is between forty centimeters and one meter, preferably between forty and sixty centimeters.

[0031] The enveloping box 30 has as its axis of revolution an axis A passing through the foot point 50 and through a central point C located at the center of the upper base BS.

[0032] If the head point 51 and the foot point 50 are on a vertical axis relative to the ground of the image, then the central point C and the head point 51 are the same. This is the case if the element of interest P is a person standing upright.

[0033] If the head point 51 and the foot point 50 are not on a vertical axis relative to the ground of the image, then the central point C and the head point 51 are distinct. This is the case if the element of interest P is a person standing for example with the head inclined towards a shoulder.

[0034] An orientation is associated with the element of interest P.

[0035] According to one example, the orientation of the element of interest P is characterized from the characteristic points defining said element of interest P.

[0036] For example, in the case of an element of interest P being a person, the orientation is characterized according to a calculation based on the pair of points relating to the shoulders (52a, 52b) and at least one other pair of points among the pair of points relating to the elbows (53a, 53b) and the pair of points relating to the wrists (54a, 54b).

[0037] Depending on the location of said pairs of points (52a, 52b), (53a, 53b), (54a, 54b) the orientation of the element of interest can be determined.

[0038] According to another example, the orientation of an element of interest P is characterized from a three-dimensional mesh of the element of interest P.

[0039] According to another example in which the element of interest P is a person or an animal, the orientation is characterized as a function of the direction of gaze of the element of interest P, for example by means of a dedicated neural network.

[0040] According to another example, the orientation of an element of interest P is characterized by a speed vector associated with the element of interest P, generated from a tracking of the movement of the element of interest P in several images acquired at different times in time,

[0041] In [Fig. 1], the orientation is represented in the image by an orientation vector 10.

[0042] For example, as shown in [Fig.l], the orientation vector 10 has as its origin the central point C and is included in the plane of the upper base BS. In this example, the orientation vector 10 is a radius of the upper base BS which points towards the circle of the disc delimiting said upper base BS.

[0043] A membership space 20 is associated with the element of interest P. The membership space 20 is represented for example in the ground plane of the environment of the element of interest of the image.

[0044] The membership space 20 of an element of interest P is defined as a function of an anchor point PA referenced relative to the element of interest P.

[0045] The membership space 20 of an element of interest P is defined as a function of a reference vector D parallel to the orientation vector 10 and in the same direction, and passing through the anchor point PA.

[0046] The anchoring point PA is advantageously a characteristic point of the element of interest P or is defined in relation to characteristic points of the element of interest P.

[0047] According to the example shown in [Fig.l], the anchor point PA is the foot point 50.

[0048] The membership space 20 comprises limits L1, L2 which are straight segments intersecting at the anchor point PA and forming a salient angle alpha.

[0049] Advantageously, the reference vector D is a bisector of said angle alpha formed by the membership space 20.

[0050] The angle alpha being less than 180 degrees and centered on a reference vector D parallel to the orientation vector 10 and of the same direction, the membership space 20 is oriented like the element of interest P.

[0051] The belonging space 20 is preferably located in the general plane of the ground of the environment of the element of interest P.

[0052] The belonging space 20 is preferably located in the plane comprising the lower base BI of the enveloping box 30.

[0053] The membership space 20 is a portion of a disk. For example, in the case of a membership space 20 comprising limits L1, L2 which are straight segments passing through the anchor point PA and forming an angle alpha of 60 degrees, the portion of the disk corresponds to one sixth of a whole disk.

[0054] The membership space 20 is delimited between the limits L1, L2 by a border B which is a portion of a circle whose origin is the anchor point PA.

[0055] The belonging space 20 is advantageously divided into a plurality of zones 21, 22, 23.

[0056] Each zone 21, 22, 23 is delimited from another zone by a boundary F1, F2, which is a portion of a circle whose origin is the anchor point PA.

[0057] Advantageously, the portions of the circles associated with the boundaries F1, F2 and the border B are concentric at the anchor point PA.

[0058] The boundaries F1, F2 and the border B each represent a predetermined distance to the anchor point PA and therefore to the element of interest P.

[0059] For example, the membership space 20 comprises a first boundary Fl corresponding to a first distance between thirty and seventy centimeters from the anchor point PA, in particular fifty centimeters. For example, the first Fl corresponds to a portion of the circle delimiting the lower base BI of the enveloping box 30, as shown in [Fig.l].

[0060] For example, the membership space 20 comprises a second boundary F2 corresponding to a second distance of between eighty and one hundred and twenty centimeters from the anchoring point PA, in particular one hundred centimeters.

[0061] For example, the belonging space 20 is delimited by the border B corresponding to a third social distance between one hundred and thirty and one hundred and seventy centimeters from the anchor point PA, in particular one hundred and fifty centimeters.

[0062] The first distance is less than the second distance, which is itself less than the third distance.

[0063] [Fig.2] represents an image comprising a first element of interest PI, a second element of interest P2 and a third element of interest P3 which are people.

[0064] The elements of interest PI, P2, P3 are represented as described previously. Everything described for [Fig.l] applies to [Fig.2].

[0065] Each element of interest PI, P2, P3 is represented by a skeleton and is defined by characteristic points.

[0066] Each element of interest PI, P2, P3 is further modeled by an enveloping box 30.

[0067] An orientation represented by an orientation vector 10 is associated with each element of interest P.

[0068] A membership space 20 originating at an anchor point PA is associated with each element of interest P.

[0069] As described for [Fig.l], each membership space 20 is divided into three zones 21, 22, 23: - a first zone 21 extending from the anchoring point PA to the first boundary F1 corresponding to the first distance from the anchoring point PA, - a second zone 22 extending from the first boundary F1 corresponding to the first distance from the anchoring point PA, to the second boundary F2 corresponding to the second distance from the anchoring point PA, - a third zone 23 extending from the second border F2 corresponding to the second distance from the anchor point PA, to the border B corresponding to the third distance from the anchor point PA.

[0070] In [Fig.2], the boundaries F1, F2 and the border B are shown but are not referenced so as not to make the figure cumbersome. The references are as shown in [Fig. 1].

[0071] The membership spaces 20 of the first and second elements of interest PI, P2 intersect at an intersection. The membership space 20 of the third element of interest P3 does not intersect any other membership space 20

[0072] An intersection between two membership spaces 20 corresponds to a connection between the two corresponding elements of interest.

[0073] A connection between two elements of interest, when confirmed over time, corresponds to an interaction between said elements of interest.

[0074] In [Fig.2], the first and second elements of interest PI, P2 are connected because their membership spaces 20 intersect The membership spaces 20 of the first and second elements of interest PI, P2 intersect. The third element of interest P3 is not connected to another element of interest.

[0075] According to one embodiment, a connection class is assigned based on a location of intersection of the zones 21, 22, 23 of the two belonging spaces 20.

[0076] For example, particularly for elements of interest which are people, we distinguish three classes of connection: intimate, personal, social.

[0077] Three conditions are defined below.

[0078] If the first two zones 21 of two membership spaces 20 of two distinct elements of interest intersect, that is to say, they overlap, then a first condition is fulfilled.

[0079] If a second zone 22 of the membership space 20 of an element of interest intersects with any of the first and second zones 21, 22 of the membership space 20 of another element of interest, then a second condition is fulfilled.

[0080] If a third zone 23 of the membership space 20 of an element of interest intersects with any of the first, second and third zones 21, 22, 23 of the membership space 20 of another element of interest, then a third condition is fulfilled.

[0081] If the first condition is met, then the connection between the corresponding elements of interest is intimate.

[0082] If the first condition is not met and the second condition is met, then the connection between the corresponding elements of interest is personal.

[0083] If the first and second conditions are not met and the third condition is met, then the connection between the corresponding elements of interest is social.

[0084] In [Fig.2], with regard to the first and second elements of interest PI, P2, the first two zones 21 of the membership spaces 20 of the two distinct elements of interest PI, P2 do not overlap, the first condition is not fulfilled.

[0085] The second zone 22 of the membership space 20 of an element of interest PI, P2 does not intersect with any of the first and second zones 21, 22 of the membership space 20 of another element of interest P2, PI, the second condition is not fulfilled.

[0086] The third zone 23 of the membership space 20 of an element of interest PI, P2 intersects with any one of the first, second and third zones 21, 22, 23 of the membership space 20 of another element of interest P2, PL. Indeed, the third zone 23 of the membership space 20 of the first element of interest PI intersects with the second and third zones 22, 23 of the membership space 20 of the second element of interest P2. Furthermore, the third zone 23 of the membership space 20 of the second element of interest P2 intersects with the second and third zones 22, 23 of the membership space 20 of the first element of interest PL. The third condition is fulfilled twice.

[0087] Thus, for the first and second elements of interest PI, P2 of [Fig.2], the first and second conditions are not met and the third condition is met. The class of the connection between the first and second elements of interest PI, P2 is social.

[0088] According to another embodiment, a connection class is assigned according to a proportion of intersection of the zones 21, 22, 23 of the two membership spaces 20 or a proportion of intersection of the two membership spaces 20.

[0089] Similarly, we distinguish three classes of connection: intimate, personal, social.

[0090] For example, weighting coefficients are applied to the proportion based on zones 21, 22, 23 to determine the connection class.

[0091] Whatever the embodiment, if the connection between the first and second elements of interest PI, P2 is confirmed over time, then there is interaction between said elements of interest PI, P2.

[0092] For example, an interaction between two elements of interest is detected if a predetermined number of connections between said two elements of interest is detected over a predefined number of successive images in time.

[0093] For example, over ten successive images acquired at a predetermined time period of between 200 and 2000 ms, if a connection between two elements of interest is detected over eight of the ten images, then an interaction is detected between said two elements of interest.

[0094] Like connection classes, and regardless of how they are determined, an interaction can be of different types, for example: intimate, personal, social.

[0095] The type of interaction provides information on the relationship and / or proximity between the elements of interest.

[0096] For example, the type of interaction is determined based on the majority class of connections detected over the predefined number of successive images over time.

[0097] A confirmation over time of a majority social class connection corresponds to a social type interaction.

[0098] A confirmation in time of a majority intimate class connection corresponds to an intimate type interaction.

[0099] A confirmation in time of a majority personal class connection corresponds to a personal type interaction.

[0100] According to the previous example, if out of the eight images, six are of intimate class and two are of personal class, the interaction is of intimate type, because the intimate class is in the majority of the detected connections.

[0101] For example, an interaction between two elements of interest PI, P2 is represented on the image by a line 40 connecting the anchor points PA of said elements of interest PI, P2, as represented in [Fig.2].

[0102] For example, the type of interaction is represented by a color on the image.

[0103] According to a non-limiting example, the color yellow is assigned for the social type, the color orange is assigned for the personal type and the color red is assigned for the intimate type.

[0104] An element of interest can be connected to several elements of interest.

[0105] An element of interest may interact with several elements of interest.

[0106] The anchor point PA of an element of interest can be connected to several anchor points PA of several elements of interest, for example by several lines 40.

[0107] The interactions between an identified element of interest and other elements of interest may be of different types. For example, an element of interest may be in social interaction with a first other element of interest and in intimate interaction with a second other element of interest.

[0108] An intersection between two belonging spaces 20 can be highlighted on the image, for example by colored, textured, hatched filling of the intersection, or by highlighting, color or line thickness of a contour of the intersection, or even by flashing the surface or contours of the intersection.

[0109] Figures 1 and 2 illustrate examples of representation of an element of interest such as a person. The skeleton, enveloping box, and characteristic point models are not limiting for carrying out the method according to the invention.

[0110] [Fig.3] illustrates the steps of the method for detecting interactions between several elements of interest, implemented by a computer.

[0111] The computer has a multitude of images comprising elements of interest. The multitude of images is acquired at different times in time.

[0112] For example, the multitude of images corresponds to images from a video acquired by a camera filming said elements of interest.

[0113] The calculator implements the method on a plurality of images.

[0114] According to one example, the plurality of images corresponds to the multitude of images acquired by the camera.

[0115] According to another example, the plurality of images corresponds to a sampling of images in the multitude of images acquired by the camera, for example to a sampling carried out at an image frequency of between 5 images per second and 50 images per second. This has the advantage for the computer of limiting the calculations and the memory resources required.

[0116] For example, for a multitude of images acquired at a frame rate of 60 frames per second by a camera, the computer samples images at a frame rate of 10 frames per second.

[0117] For each image i of the plurality of images, the method comprises a characterization step 101, a definition step 102 and an optional display step 103.

[0118] According to the characterization step 101, the calculator characterizes for each element of interest, an orientation of the element of interest.

[0119] For example, the orientation is characterized from a three-dimensional mesh of the element of interest or as a function of the direction of a gaze.

[0120] For example, the orientation of an element of interest P is characterized by a velocity vector associated with the element of interest P.

[0121] For example, the orientation is characterized from characteristic points defining the element of interest.

[0122] According to a selection step 100 prior to the characterization step 101, the calculator selects characteristic points defining the element of interest.

[0123] For example, when the element of interest is a person and as illustrated [Fig.l], the calculator selects a pair of points relating to shoulders (52a, 52b), and at least one other pair of points from among a pair of points relating to wrists (54a, 54b) and a pair of points relating to elbows (53a, 53b).

[0124] Then, according to the characterization step 101, the calculator characterizes the orientation according to a calculation based on said pair of points relating to the shoulders (52a, 52b) and at least one other pair of points (53a, 53b), (54a, 54b).

[0125] For example, the calculator characterizes the orientation according to a calculation based on the three pairs of points (52a, 52b), (53a, 53b), (54a, 54b) weighted by weights, a first weight being attributed to the pair of points relating to the shoulders (52a, 52b), a second weight being attributed to the pair of points relating to the elbows (53a, 53b) and a third weight being attributed to the pair of points relating to the wrists (54a, 54b), the first weight being higher than the second weight, the second weight being higher than the third weight.

[0126] For example, the calculator characterizes the orientation using the image coordinates of the pairs of points in image i and the corresponding real coordinates in the real world.

[0127] Advantageously, a weighting coefficient is applied to the image coordinates of the pairs of points (52a, 52b), (53a, 53b), (54a, 54b) as a function of the posture of the person.

[0128] Indeed, when the person is positioned with a front bust in the image, it is advantageous to give more weight to the image coordinates than to the real coordinates, the image coordinates being more precise for determining the orientation. When the person is positioned with a profile bust in the image, it is on the contrary advantageous to minimize the weight given to the image coordinates for determining the orientation.

[0129] Advantageously, the calculator determines an orientation vector 10 as shown in FIGS. 1 and 2, comprising an orientation direction and an orientation sense.

[0130] According to the definition step 102, the calculator defines for each element of interest, a membership space 20 as a function of the orientation of the characterized element of interest.

[0131] For example, as shown in Figures 1 and 2, the membership space 20 of an element of interest P, PI, P2, P3 is defined as a function of an anchor point PA referenced relative to the element of interest, and of a reference vector D passing through the anchor point PA, parallel to the orientation vector 10 and in the same direction.

[0132] As described for [Fig. 1], when the element of interest P is a person, the anchor point PA is for example a foot point 50, located in the middle of a left foot point and a right foot point of said person.

[0133] The membership space 20 comprises limits L1, L2 passing through the anchor point PA and forming a salient angle alpha of which the reference vector D is a bisector and the anchor point PA the vertex.

[0134] Such an example is illustrated in [Fig.l]. The limits L1, L2 are segments which intersect at the anchor point PA, forming an angle alpha with the vertex at the anchor point PA and the side limits L1, L2.

[0135] For example, the value of the angle alpha is between 45 degrees and 120 degrees, preferably between 45 degrees and 75 degrees.

[0136] As the reference vector D is a bisector of the angle alpha and is of the same sense and direction as the orientation vector 10, the membership space 20 is centered and oriented according to the orientation of the associated element of interest.

[0137] The belonging space 20 is preferably located in the general plane of a ground of the environment of the element of interest.

[0138] According to the optional display step 103, the computer displays on a screen superimposed on the image i a delimitation of the belonging space 20; for example as represented in FIGS. 1 and 2.

[0139] For example, the calculator also displays superimposed on the image i a representation of the orientation vector 10 associated with the element of interest P, PI, P2, P3, as represented in Figures 1 and 2.

[0140] For example, the calculator also displays superimposed on the image a representation of an enveloping box 30 associated with the element of interest P, PI, P2, P3, as shown in FIGS. 1 and 2.

[0141] In the case where the plurality of images i on which the characterization 101, definition 102 and display 103 steps are implemented corresponds to a sampling of a larger number of images acquired by a camera, the calculator extrapolates the elements displayed superimposed on the plurality of images resulting from the method to the non-sampled images i0 according to an extrapolation step 110.

[0142] An extrapolation is for example the result of a transformation by translation and / or rotation. The element to be displayed in superposition is transformed by translation and / or rotation according to the displacement of the element of interest.

[0143] Thus, the computer is further capable, according to the display step 103, of displaying a non-sampled image iO without display discontinuity superimposed with a non-sampled image i.

[0144] According to a step 104 of detecting interactions, the calculator detects interactions between elements of interest PI, P2, P3 as a function of the relative location of the membership spaces 20 of the elements of interest PI, P2, P3 with respect to each other on several images i in time.

[0145] For example, the interaction detection step 104 comprises determining intersections between the membership spaces 20 of the elements of interest PI, P2, P3, an intersection between two membership spaces 20 corresponding to a connection between the two elements of interest associated with said two membership spaces 20, an interaction being detected as a function of several connections over time between said two elements of interest.

[0146] Examples and details on the determination of intersections have been described with reference to [Fig.2] and apply to this detection step 104.

[0147] Advantageously, as described for [Fig.2], each membership space 20 is divided into a plurality of zones 21, 22, 23. A connection class between two elements of interest is assigned by the calculator as a function of a location or a proportion of intersection of the zones 21, 22, 23 of the two membership spaces 20.

[0148] Examples and details of the assignment of connection classes have been described with reference to [Fig.2] and apply to the method.

[0149] An interaction is detected by the calculator if a predetermined number of connections between said two elements of interest is detected over a predefined number of successive images i in time.

[0150] For example, for a plurality of images i separated in time by 100 ms, the predefined number of images is advantageously between 5 and 20.

[0151] The calculator can determine types of interactions between elements of interest from the classes of connections between said elements of interest.

[0152] Examples and details have been described with reference to [Fig.2] and apply to the method.

[0153] According to the optional display step 103, the computer displays superimposed on the image a representation of the detected interaction(s), for example, by a line 40 as shown in [Fig. 2]. The type of interactions can also be displayed, for example, as described in relation to [Fig. 2].

[0154] For example, the calculator displays a representation of the detected intersections superimposed on the image, for example, as described in relation to [Fig.2].

[0155] The method according to the invention may further comprise an establishment step 105 in which the calculator establishes a list of elements of interest having entered into interaction with a predetermined element of interest as a function of detected interactions.

[0156] For example, the list includes the type of interactions detected.

[0157] In Figures 1 and 2, the elements of interest are pedestrians. This is a non-limiting example of elements of interest. The method of the invention is applicable to people in vehicles, for example on a scooter, bicycle, motorcycle, car, to animals or to intelligent objects such as robots or autonomous vehicles. [Fig. 4] illustrates a system comprising a computer 202 capable of implementing the method of the invention, a detector 203 of elements of interest, a display screen

[0158]

[0159]

[0160]

[0161]

[0162]

[0163]

[0164]

[0165] 204 and a shooting device 200 capable of acquiring images i, iO. The shooting device 200 is oriented along an axis 201 towards a region of an environment comprising elements of interest P. Image i shown in [Fig.4] includes an element of interest P. The element of interest P is defined by characteristic points. In image i, a characteristic point is called image characteristic point ki and has image coordinates (x,y) in a two-dimensional image frame with predetermined axes Ax, Ay. In the real world of the shooting device 200, said corresponding characteristic point is called real characteristic point Ki and has real coordinates (X, Y, Z) in a predetermined three-dimensional real frame of reference with axes AX, AY, AZ. The detector 203 is capable of detecting elements of interest P and modeling them, for example, into skeletons of fifteen points when the elements of interest P are people. The detector 203 is further capable of determining the image coordinates of the skeleton points in the image i and of estimating the real coordinates of the skeleton points in the real world. The display screen 204 is capable of displaying images i, iO acquired by the shooting device 200 as well as superimposed display elements, such as an orientation vector, a delimitation of a belonging space, an enveloping box, a representation of detected interactions, etc. The calculator 202, from the calibration parameters of the shooting device 200, knows the function [x 1 , \ which allows, from the image coordinates of a J \lyr n / image point ki [ X1 expressed in pixels and of estimated height h expressed in centimeters, m the real coordinates of the corresponding real point Ki X, expressed in centimeters, y z. with Y=h.

[0166]

[0167]

[0168]

[0169]

[0170] The 202 calculator also knows the inverse function / X allowing to LZJ / find the image coordinates of an image point ki [X1 from the coordinates LJJ real values ​​of the corresponding real point Ki [XI. Y .Z. In such a system, the calculator 202 can characterize the orientation of an element of interest P by determining an orientation vector v. The calculator 202 selects characteristic points beforehand, for example from the skeleton points provided by the detector 203: - a pair of points relating to the shoulders, - a pair of points relating to the elbows, - a pair of points relating to the wrists, - a pair of points relating to the spine, for example with one point relating to the pelvis and one point relating to the thorax, - a point relating to the head, - a point relating to the feet. For example, the orientation vector v is as shown in Figures 1 and 2 and is determined according to the following mathematical formula: v = ). with: - / îp being respectively the weight attributed to the pair of points relating to the shoulders, the weight attributed to the pair of points relating to the elbows, the weight attributed to the pair of points relating to the wrists, with for example ^^1, = 0.5, = 0.25, -Al being the real coordinates of a vector relative to the two shoulders in the real world, - al being the image coordinates of a vector relative to the two shoulders in image i, - A2 being the real coordinates of a vector relative to the two elbows in the real world, - a2 being the image coordinates of a vector relative to the two elbows in image i, - A3 being the real coordinates of a vector relative to the two wrists in the real world, - a3 being the image coordinates of a vector relative to the two wrists in image i, - PT being the real coordinates of a vector relative to the spine, - ® being the vector product operator, -g being a function defined by: - gl being a function defined by - R being the predetermined norm of the orientation vector, equal to the radius of the upper base BS of an enveloping box, for example equal to 50 centimeters. - CP being a weighting coefficient applied to the image coordinates of the pairs of points, - kte being a value representative of a real distance between two shoulders of an adult person of average height, for example 50 centimeters, - ||a-|| being the norm of the vector relative to the shoulders in image i and affecting the weight of the image coordinates of the pairs of points in the calculation of the orientation vector, depending on the posture of the person, - hp being the estimated height of the person, for example determined from a point relative to the feet and a point relative to the head, - C being a head point or more advantageously the central point C as described previously, in relation to figure 1.

[0171] A person skilled in the art will know how to adapt this mathematical formula to take into consideration only the pair of points relating to shoulders and at least one other pair of points among a pair of points relating to wrists and a pair of points relating to elbows or else take into account an additional pair of points, for example the hands.

[0172] It is also possible to take into account only a pair of points to characterize the orientation of an element of interest, for example the pair of points relating to shoulders or the pair of points relating to wrists, but the result will be less reliable.

[0173] According to the examples illustrated in the figures and described above, the elements of interest are pedestrians. Those skilled in the art will know how to adapt the example for elements of interest which are people in vehicles.

[0174] For example, for an element of interest which is a person in a vehicle, the orientation can be determined in the same way as for a pedestrian or take into account the orientation of the vehicle, the belonging space can take into account a volume occupied by the vehicle, the anchor point can be located on the vehicle or the person.

[0175] A person skilled in the art will know how to adapt the example of the element of interest which is a pedestrian to an element of interest which is an animal. An animal can be modeled by a skeleton, just like a person.

[0176] A person skilled in the art will know how to adapt the example of the element of interest which is a pedestrian to an element of interest which is a robot or an autonomous vehicle. For example, an element of interest which is a robot or an autonomous vehicle comprises a control unit. The orientation can be determined using data provided by the control unit of the element of interest to the computer, the data being for example trajectory or geographical position data. More similarly to the examples described, the orientation can be determined from characteristic points of the element of interest.

Claims

Claims

1. Method for detecting interactions between several elements of interest (PI, P2, P3), by a computer, from a plurality of images of a video acquired by a camera filming said elements of interest, the plurality of images comprising the elements of interest, the images of the plurality of images being acquired at different times in time, the method comprising for each image the following steps: - a characterization step (101), in which the computer characterizes for each element of interest, an orientation of the element of interest, - a definition step (102), in which the computer defines for each element of interest, a membership space (20) as a function of the orientation of the characterized element of interest, and comprising a step of detection (104) of interactions, in which the computer detects interactions between elements of interest (PI, P2,P3) depending on the relative location of the membership spaces (20) of the elements of interest (PI, P2, P3) in relation to each other in time.,

2. Method for detecting interactions according to the preceding claim, at least one element of interest being a person, the method for detecting interactions comprising, for each element of interest being a person, a selection step (100) in which the calculator selects characteristic points defining the element of interest (PI, P2, P3), the points comprising a pair of points relating to shoulders (52a, 52b), and at least one other pair of points from among a pair of points relating to wrists (54a, 54b) and a pair of points relating to elbows (53a, 53b), the orientation being characterized according to a calculation depending on the pair of points relating to the shoulders (52a, 52b) and the at least one other pair of points (53a, 53b, 54a, 54b).

3. Method for detecting interactions according to the preceding claim, the calculator selecting the three pairs of points comprising the pair of points relating to shoulders (52a, 52b), the pair of points relating to wrists (54a, 54b) and the pair of points relating to elbows (53a, 53b), the orientation being characterized according to a calculation based on the three pairs of points (52a, 52b, 53a, 53b, 54a, 54b) and weighted by weights, a first weight being assigned to the pair of points relating to the shoulders (52a, 52b), a second weight being assigned to the pair of points relating to the elbows (53a, 53b) and a third weight being assigned to the pair of points relating to the wrists (54a, 54b), the first weight being higher than the second weight, the second weight being higher than the third weight.

4. Method for detecting interactions according to any one of the preceding claims, the orientation comprising an orientation vector (10), the membership space (20) of an element of interest being defined as a function of an anchor point (PA) referenced relative to the element of interest and a reference vector (D) passing through the anchor point, parallel to the orientation vector (10) and in the same direction, the membership space (20) comprising limits (L1, L2) passing through the anchor point (PA) and forming a salient angle (alpha) of which the reference vector (D) is a bisector and the anchor point (PA) the vertex.

5. Method for detecting interactions according to any one of the preceding claims, the step of detecting (104) interactions comprising the determination of intersections between the membership spaces (20) of the elements of interest (PI, P2, P3), an intersection between two membership spaces (20) corresponding to a connection between the two elements of interest associated with said two membership spaces (20), an interaction being detected as a function of several connections in time between said two elements of interest.

6. Method for detecting interactions according to the preceding claim, each membership space (20) being divided into a plurality of zones (21, 22, 23), a connection class between the two elements of interest being attributed according to a location of intersection of the zones (21, 22, 23) of the two membership spaces (20).

7. Method for detecting interactions according to any one of claims 5 to 6, an interaction being detected if a predetermined number of connections between said two elements of interest is detected over a predefined number of successive images in time.

8. A method of detecting interactions according to any preceding claim, further comprising a step establishing (105) a list in which the calculator establishes a list of elements of interest (PI, P2, P3) having entered into interaction with a predetermined element of interest as a function of detected interactions.

9. Method for detecting interactions according to any one of the preceding claims, further comprising for each image, a display step (103) in which the calculator displays superimposed in the image, for each element of interest, a delimitation of the membership space (20) and a representation of the interactions detected if interactions have been detected.

10. A computer program product comprising the program instructions implementing the steps of the interaction detection method according to any one of claims 1 to 9, when the program instructions are executed by a computer.

Images